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1.0  INTRODUCTION 


1.1  Background 

The  Aids  to  Navigation  Positioning  Project  (AN PP)  was  initiated  in 
the  Coast  Guard  to  establish  ways  to  improve  the  reliability  of  buoys  in 
marking  advertised  positions.  Support  efforts  in  this  task  were  assigned  to 
the  CG  Research  and  Development  Center  at  Groton,  Connecticut,  and  are 
entitled  "The  Aids  to  Navigation  Position  Accuracy  and  Reliability  (ANPAR) 
Project." 


1.1.1  Philosophy 

The  Positioninq/Error  Model  has  been  defined  in  three  dis 

tinct  phases: 


I  -  Error  Sensitivity  Model 
II  -  Operational  Positioning  Model 
III  -  Uses  of  the  Models 

This  report  concerns  itself  with  Phase  I  which  adopted  the 
following  philosophy: 


Error  sources,  both  random  and  systematic  in  nature,  are 
inherent  in  the  positioning  process.  Each  error  source  can  cause  an  error  in 
one  or  more  of  the  observations  from  which  position  is  determined.  The 
resulting  observation  error  will  be  propagated  within  the  positioning  process 
and  result  in  a  corresponding  position  error.  This  error  propagation  Is  a 
function  of  both  the  fix  geometry  (locations  of  landmarks  or  transmitting 
stations)  and  the  observed  parameter  (sextant  angle,  radar  range,  gyroscope 
bearing,  or  LORAN  time  differences).  It  is  reasonable  to  construct  a 
mathematical  model  of  error  propagation  routines  which  can  include,  where 
practical,  the  functional  relationship  between  a  measure  of  error  source 
magnitude  and  a  corresponding  observation  error.  For  the  purpose  of  this 
report  the  term  "error"  will  refer  to  either  an  error  source  magnitude  or  an 
observation  error;  the  effect  of  an  "error"  is  called  "position  error";  and 
the  error  propagation  routine  is  called  "modeled  error."  Errors  with  similar 
propagation  functions  can  be  grouped  and  referred  to  by  a  representative 
"modeled  error"  to  avoid  redundant  model  routines. 

Studies  of  error  in  the  positioning  process  can  be 
conducted  by  constructing  a  model  which  can  accommodate  various  modeled 
errors,  error  magnitudes,  and  fix  geometries,  as  well  as  display  various 
measures  of  position  error.  The  model  developed  by  this  project  allows  the 
user  to:  (1)  select  the  modeled  error  to  be  studied;  (2)  specify  the  range  of 
error  magnitude;  (3)  either  define  one  particular  geometry  or  specify  the 
number  of  randomly  generated  geometries  for  an  average  effect;  and  (4)  select 
the  position  error  effect  to  be  displayed. 

Studies  using  this  model  of  the  positioning  process  can 
be  conducted  for  ranking  errors  by  their  effect  on  a  selected  position  error 
measure.  Once  errors  are  ranked,  efforts  can  be  directed  toward  minimizing 
the  magnitudes  of  the  priority  errors.  This  model  also  can  be  used  to  study 
the  effect  blunders  have  on  the  positioning  process.  The  most  significant 
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blunder  effects  can  be  Identified  and  examined  to  ascertain  the  need  for 
training  efforts  directed  towards  amelioration  of  these  effects. 

The  Error  Sensitivity  Model  (ESM)  phase  of  this  effort 
provides  Insight  into  the  effect  of  the  various  error  sources  and  provides  a 
tool  for  use  by  ANPP  to  support  further  efforts.  The  result  of  such  efforts 
should  help  Coast  Guardsmen  comply  with  the  requirement  that  an  aid  mark 
accurately  and  precisely  the  geographic  position  advertised  by  the  govern¬ 
ment.  In  this  light,  the  Error  Sensitivity  Model  is  a  management  tool  to  help 
study  the  aid  positioning  macro-structure. 

1.2  Terminology  and  Symbology 

The  terminology  established  by  reference  1  has  been  adopted  here  and 
extended  as  necessary.  In  addition  to  the  Glossary  of  reference  1,  the 
following  publications  establish  the  terminology  used  herein: 

a.  American  Practical  Navigator.  Bowditch  (1977) 

b.  Aids  to  Navigation  Manual.  Positioning.  Volume  V,  1978 

c.  Observations  and  Least  Sguares,  Mikhail.  E.M. 

d.  Hydrographic  Manual,  4th  Edition,  Department  of  Commerce 

e.  Definitions  of  Survey  Terms,  Mitchell,  H.C.,  Special  Pub.  No. 

232 

Symbols  are  defined  where  they  are  first  used  in  the  text. 

1.3  Summary  of  Previous  Work 

A  review  of  the  descriptive  bibliography  of  reference  1  indicates 
that  ANPP  work  related  to  the  ESM  has  been  in  progress  for  about  seven  years. 
Items  listed  in  this  bibliography  (along  with  some  unpublished  notes  compiled 
within  the  Coast  Guard)  comprise  the  majority  of  previous  work  on  the  ESM 
Phase  of  the  Positioning/Error  Model. 
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2.0  MATHEMATICAL  M00SL 


The  model  simulates  the  Inherent  stochastic  nature  of  observations 
through  use  of  systematic  and  random  error  inputs  making  possible  the 
observation  of  error  propagation  effects  on  important  mathematical  measures  of 
position  error. 

The  model  employs  the  least  squares  principle  to  find  a  set  of  adjust¬ 
ments  that  will  cause  the  observations  to  be  compatible  with  the  following 
functional  model: 

-  Within  the  local  region  where  the  positioning  process  is  being  exe¬ 
cuted,  the  lines  of  position  determined  by  any  sufficient  subset  of 
functionally  independent  observations  should  cross  at  a  single  point. 

2.1  Preliminary  Considerations 

Mathematical  models,  by  necessity,  must  rely  upon  appropriate 
assumptions  and/or  approximations.  When  these  are  understood,  the  model  is 
applicable  to  the  domain  defined  by  them.  These  considerations  are  defined 
and  expanded  in  this  and  succeeding  sections  of  the  report.  While  every 
effort  has  been  made  to  select  error  magnitudes  that  reflect  those  occurring 
in  practice,  the  model  is  flexible  enough  to  incorporate  verified  values  as 
they  become  available. 

While  the  basic  transformation  from  observation  space  to  real 
positioning  space  is  well  known,  the  statistical  methods  chosen  for  weighting 
each  observation  and  simulating  each  error  source  in  the  system  are  dictated 
by  the  project  element  requirements.  The  requirements  of  phase  I  of  this 
project  element  are  to  both  model  and  study  position  error  effects  as  a 
function  of  the  systematic  and  the  random  measurement  error  components. 
Statistics  which  effectively  deal  with  each  of  these  error  components  are  a 
necessity. 


The  basic  tenet  (reference  1),  "that  the  conditions  Imposed  upon  any 
positioning  task  are  unique  to  the  station  and  not  necessarily  well  defined," 
dictate  procedures  which  statistically  account  for  the  uncertainty  of  a 
specific  positioning  evolution  by  calculating  unbiased  estimates  of 
measurement  precision  from  observations  made  on  scene.  These  procedures  deal 
with  the  total  measurement  error,  not  its  components,  and  are  not  needed  to 
satisfy  the  requirements  of  phase  I  of  this  project  element.  These  procedures 
will  be  discussed  in  phase  II  of  the  Positioning/Error  Model  effort  where 
emphasis  is  placed  on  mathematical  tools  for  actual  aid  positioning.  (See 
reference  1) 

The  results  of  ESM  studies  presented  In  this  report  are  position 
error  effects  as  a  function  of  error  magnitudes  and  provide  an  Indication  of 
which  error  to  prioritize.  The  measurement  and  quantification  of  actual  error 
magnitudes  Is  the  subject  of  other  project  elements  of  the  ANPAR  project. 
Definite  error  magnitudes  can  be  Input  to  the  model  for  analysis  when  they  are 
established. 

2.2  Least  Squares  Adjustment  Method 

The  error  model  Is  based  on  the  least  squares  adjustment  method  of 
computing  the  Most  Probable  Position  (MPP).  Mathematical  justification  for 
the  adjustment  method  Is  contained  In  references  4  and  8.  Expressions  for 
each  matrix  element  are  provided  in  appendix  A. 
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The  least  squares  method  is  used  to  provide  probable  corrections  to 
a  set  of  observations  so  that  they  will  satisfy  the  functional  model.  The 
input  is  a  set  of  inconsistent  measurements  with  an  inherent  stochastic  nature 
and  the  output  is  an  adjusted  set  of  measurements  which  are  consistent  but  are 
not  the  true  measurements.  In  a  related  fashion,  the  least  squares  method  can 
be  used  to  study  the  effect  observation  errors  have  on  position  error. 

A  "fix  geometry"  is  defined  so  that  the  set  of  errorless  measure¬ 
ments  cause  a  consistent  set  of  lines  of  position.  The  observations  are  then 
subjected  to  a  known  error,  or  set  of  errors,  and  the  resulting  inconsistent 
system  is  studied.  If  the  system  is  subjected  to  systematic  error,  the  MPP 
may  change  its  location  and  that  change  will  be  dependent  on  the  weight  of  the 
observations  subjected  to  the  error  and  the  structure  of  the  fix  geometry. 
Mathematically,  displacement  of  the  MPP  in  positioning  space  is  defined  by  the 
vector  XT  =  [ax.  Ay]  .  The  vector  X  is  determined  through  the  transfor¬ 
mation  of  n  observations  given  by:  (references  3  and  8) 


x  *  -(aT'waj-Wwl 


(2-1) 


Where 


L  is  an  (n  x  1)  vector  matrix  of  systematic  observation  errors. 

W  is  an  (n  x  n)  diagonal  matrix  of  observation  weighting  factors 
(inverse  of  covariance  matrix). 

A  is  an  (n  x  2)  matrix  of  partial  derivatives  of  measurements  with 
respect  to  x  and  y  dimensions  in  real  positioning 
space . 


It  is  upon  this  equation  that  the  Error  Sensitivity  Model  is  based. 
The  A  matrix  is  completely  determined  by  the  "fix  geometry"  specified  for 
modeling.  The  L  matrix  is  the  tool  for  systematic  error  input  to  the  system 
and  the  W  matrix  is  the  weighting  matrix  which  depends  on  the  specified  vari¬ 
ances  (and  therefore  weight)  of  each  observation.  It  can  be  seen  at  this 
point  that  the  effect  a  systematic  error  has  on  X  is  linear  but  is  dependent 
heavily  on  the  variance  of  the  measurements  to  which  it  is  subjected. 


The  MPP  displacement  vector  X  is  not  the  only  error  effect  of 
interest.  The  stochastic  nature  of  the  observations  is  transformed  into  a 
bivariate  normal  probability  distribution  (reference  11)  centered  around  the 
MPP.  Selected  parameters  of  this  distribution  are  calculated  as  a  function  of 
the  stochastic  nature  of  the  observations. 


The  next  step  is  to  define  the  error  sources  within  the  positioning 
process  and  to  functionally  relate  each  in  turn  to  the  respective  elements  of 
either  the  L  or  W  matrix.  In  this  fashion,  the  errors  of  interest  can  be 
studied  for  any  "fix  geometry." 

2.3  The  Errors  Modeled 


Each  element  of  the  l  matrix  in  the  adjustment  equation  (2-1) 
is  a  function  of  the  systematic  error  sources  present  in  making  the  i*h 
observation.  In  an  actual  positioning  evolution  each  1*  is  subject  to 
random  error  sources  present  in  making  a  measurement.  Because  of  the  random 
nature  of  If,  each  element  is  associated  with  the  variance  of  the  respective 
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measurement  type  which  is  used  to  weight  the  observations.  The  diagonal 
elements  w^  of  the  W  matrix  in  the  adjustment  equation  are  the  weighting 
factors,  they  are  determined  by  the  equation: 


Wii 


(2-2) 


Where  <Tj  is  the  standard  deviation  of  the  i^  observation  in  units  of  the 
observation. 

The  matrix  elements  1-j  and  w^  are  the  dependent  variables  of 
the  functional  relationships  between  errors  and  the  adjustment  equation.  For 
any  "fix  geometry,"  the  effect  that  varying  the  fundamental  elements  pi  and 
wii)  has  on  the  resulting  position  error  could  have  been  studied.  This, 
however,  does  not  make  the  relative  significance  of  the  various  error  sources 
obvious.  The  output  of  the  routine  is  a  function  of  readily  understandable 
errors  which  are  described  in  words  in  the  following  sections  and  mathe¬ 
matically  in  appendix  A. 

Throughout  this  report,  the  measuring  instruments  are  assumed  to  be 
the  sextant  (angles),  the  gyrocompass  (bearings),  the  radar  (ranges),  and  the 
LORAN  receiver  (time  differences).  These  four  measuring  instruments  were 
chosen  because  of  field  familiarity  with  the  measurement  terminology  of  each. 
The  model  is  capable  of  simulating  other  range,  bearing,  angle,  or  time  dif¬ 
ference  measuring  instruments  in  the  same  manner  it  simulates  the  four  chosen 
instruments. 


2.3.1  Modeled  Systematic  Errors 

The  ERRORS  column  of  table  2-1  lists  readily  apparent  sys¬ 
tematic  errors  subject  to  study  through  use  of  the  ESM.  Many  errors  have  a 
similar  effect  on  the  position  error.  Therefore,  errors  with  similar  effect 
are  grouped  and  listed  under  the  heading  -  MODELED  ERROR.  The  MODELED  ERROR 
column  of  table  2-1  is  a  list  of  these  groups.  Each  of  the  modeled  errors  is 
described  briefly  in  the  subsections  to  follow.  For  this  report,  an  error 
maximum  for  each  modeled  error  is  given  in  table  2-1.  The  maximum  error  must 
be  assigned  by  the  user  of  the  ESM.  Because  the  model  iterates  through  a 
range  from  zero  to  the  maximum  error,  it  is  not  necessary  to  specify  an 
expected  value  of  any  systematic  error.  It  is  accepted  that  the  list  of  sys¬ 
tematic  errors  may  not  be  entirely  complete;  but,  any  exclusions  are  of  minor 
importance,  since  any  measurement  error  omitted  may  be  grouped  with  one  of  the 
errors  listed  for  modeling. 

2. 3. 1.1  Index  Error 

All  angular  measurement  errors  which  add  directly  to 
the  angular  measure  are  grouped  with  Index  Error  for  purposes  of  this  model 
because  Index  Error  is  probably  the  best  known  error  in  making  sextant 
measurements . 
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SYSTEMATIC  ERRORS 


ERROR 


INSTRUMENT 

ERRORS 

MODELED  ERROR 

C0DE# 

MAXIMUM+ 

SEXTANT 

(Angles) 

Index 

Personal 

Three-Arm  Protractor 
Instrument 

Sextant 

Index  Error 

n-IE 

60  minutes 

Inclined  Angle 

Inclined  Angle 

Error  (C4NC)# 

n-IA 

100  meters 

Observer  Coincidence 

Observer  Coinci¬ 
dence  Error 

n-OC 

20  meters 

Survey 

Alignment 

Landmark  Charting 

Landmark  Misplaced 
Error  (C4NC)# 

n-LMM 

100  meters 

RADAR 

(Ranges) 

Radar  Range  Error 

♦Variable  Electronic  Bias 
♦Range  Marker  Alignment 
♦Personal 

♦Landmark  Misplaced 

n-RE 

100  meters 

GYROCOMPASS 
(True  Bearings) 

♦Gyro 

♦Speed 

Personal 

Repeater  Offset 
♦Landmark  Misplaced 

Bearing  Error 

n-BE 

3  degrees 

LORAN 

(Time  Differences) 

♦Prediction 
♦Electronic  Bias 

LORAN  Error 

n-LE 

0.25  /usee 

♦Transmission 


*  C  *  Common  landmark  for  several  observations. 

NC  *  No  common  landmark  to  any  two  observations. 

n  «  The  number  of  measurements  in  each  simulated  geometry  subjected  to  the  error. 

*  These  errors  were  not  modeled  in  detail,  even  though  they  may  have  a  complicated 
functional  relationship  to  the  elements  of  the  L  matrix,  because  the  weight  of 
observations  made  by  Instruments  other  than  the  sextant  is  small  compared  to  the 
weight  of  sextant  observations. 

+  This  quantity  was  arbitrarily  chosen  for  this  study. 


Personal  Error  -  This  error  occurs  when  individual 
observers  make  measurements  that,  "on  the  average,"  are  a  little  larger  or  a 
little  smaller  than  they  actually  should  be. 

Three-Arm  Protractor  Error  -  This  error  is  due  to 
inaccuracies  in  a  three-arm  protractor  which  is  used  for  plotting  or  unplot¬ 
ting  angular  values. 


Instrument  Error  -  The  sum  of  all  non-adjustable 
errors  in  a  sextant  including,  prismatic,  graduation,  and  centering  errors, 
constitute  Instrument  Error. 

Sextant  Error  -  The  total  systematic  error  in  a 
sextant  measurement  is  considered  sextant  error.  Index  error,  mirror  perpen¬ 
dicularity,  and  instrument  error  are  included  in  this  group. 

2. 3. 1.2  Inclined  Angle  Error 

When  a  sextant  is  used  to  measure  the  horizontal 
angle  subtended  by  two  landmarks,  an  error  may  occur  if  the  landmarks  are  of 
different  altitudes.  The  error  is  called  Inclined  Angle  Error.  For  purposes 
of  modeling,  the  Inclined  Angle  is  given  the  units  of  meters  which  represents 
the  difference  in  altitudes  of  the  two  landmarks.  The  functional  form  of  the 
model  for  Inclined  Angle  Error  is  provided  in  appendix  A. 

2.3. 1.3  Observer  Coincidence  Error 


When  resection  methods  are  used  in  positioning,  the 
assumption  is  normally  made  that  all  angles  are  observed  with  all  observers 
standing  on  the  same  point.  This,  however,  is  usually  not  the  case,  and 
results  in  position  error.  The  error  in  angular  measurement  due  to  a  lack  of 
coincidence  of  the  observers  is  called  Observer  Coincidence  Error.  The  orien¬ 
tation  of  the  observer  separation  with  respect  to  the  "fix  geometry"  is  also 
important.  For  modeling  purposes,  the  observer  making  the  measurement  in 
error  is  displaced  in  the  direction  that  causes  maximum  effect.  The  function¬ 
al  relationship  between  observer  coincidence  error  and  the  elements  of  the  L 
matrix  is  provided  in  appendix  A. 


2. 3. 1.4  Landmark  Misplaced  Error 

This  modeled  error  can  be  used  to  simulate  errors 
occurring  when  the  geographic  coordinates  of  a  landmark  differ  from  those  used 
in  computations  or  plotting.  Survey  Error  falls  In  this  group.  Survey  error 
represents  any  case  where  the  published  coordinates  of  the  landmark  are  not  in 
agreement  with  the  coordinates  of  the  landmark.  This  may  be  due  to  lower 
order  survey  work  or,  even  worse,  to  rebuilding  of  landmarks  in  different 
locations  without  a  new  survey. 


In  using  the  sextant  to  measure  horizontal  angles, 
two  landnarks  must  be  aligned  in  the  horizon  mirror.  If  the  two  landmarks  are 
aligned  using  points  other  than  the  published  coordinates  of  the  surveyed 
point  of  the  landmark.  Alignment  Error  will  occur.  For  example,  alignment 
error  will  occur  when  lining  up  the  perceived  centerlines  of  two  tanks  rather 
than  the  surveyed  points  on  the  platforms  around  the  tanks.  The  effect  of 


this  error  is  identical  to  survey  error  but  the  source  is  the  observer  rather 
than  the  survey  network. 


Cartography  is  the  science  or  art  of  making  maps  and 
charts.  Though  the  techniques  used  by  cartographers  are  quite  accurate,  there 
is  always  a  small  allowable  error  in  charting  any  specific  landmark.  Charting 
error  contributes  to  position  error  when  graphical  procedures  are  used  in 
positioning.  For  purposes  of  modeling,  this  small  error  is  called  Landmark 
Charting  Error  and  is  placed  in  the  Landmark  Misplaced  group. 

The  orientation  of  the  Landmark  Misplacement  error 
with  respect  to  the  "fix  geometry"  can  be  important,  as  it  is  with  Observer 
Coincidence.  For  the  purpose  of  modeling,  the  misplacement  is  always  perpen¬ 
dicular  to  the  simulated  line  of  sight  to  the  landmark.  The  mathematical 
details  of  the  Landmark  Misplaced  group  are  contained  in  appendix  A. 

2. 3. 1.5  Range  Error 

There  are  many  possible  sources  of  error  in  measur¬ 
ing  ranges  using  either  electronic  or  mechanical  devices.  Most  errors  do  not 
have  a  direct  relationship  to  the  elements  of  the  L  matrix.  Model  require¬ 
ments,  however,  do  not  specify  a  need  for  the  exact  formulation  of  each 
relationship.  All  errors  in  this  group  are  simulated  by  direct  addition  to 
the  measurement. 


Radar  Error  -  The  difference  between  radar  range 
output  and  a  true  geodetic  range  is  called  Radar  Error.  Unfortunately, 
calibration  by  using  this  comparison  is  often  not  performed;  only  an  electron¬ 
ic  "tune  up"  is  performed.  The  lack  of  an  electronic  "tune  up"  may  cause  an 
Electronic  Bias  Error  to  be  overlooked. 

Variable  Range  Marker  Alignment  Error  -  This  error 
occurs  when  the  user  does  not  correctly  align  the  variable  range  marker  with 
the  blip  thnt  represents  the  landmark.  This  error  is  expected  to  be  closely 
related  to  Personal  Error  because  most  observers  have  their  own  way  of  align¬ 
ing  the  marker. 


Landmark  Misplaced  Error  -  This  error  also  applies 
to  radar  measurements,  but  the  functional  form  for  this  error  is  used  only  in 
angular  measurement  simulation  because  of  the  relative  weights  of  the  measure¬ 
ment  types. 


2. 3. 1.6  Bearing  Error 

The  determination  of  true  bearings  using  the  gyro¬ 
compass  is  subject  to  error  from  many  sources.  Most  of  the  errors  are  related 
to  the  elements  of  the  L  matrix  in  a  complex  manner.  Model  requirements, 
however,  did  not  specify  a  need  for  the  exact  formulation  of  the  relation¬ 
ship.  All  errors  in  this  group  are  simulated  by  direct  addition  to  the  true 
bearing  measurement. 


Gyro  Error  -  The  Gyro  Error  is  the  total  combination 
of  the  gyrocompass  errors  at  any  time.  It  Is  expressed  in  degrees  east  or 
west  to  indicate  the  direction  in  which  the  axis  of  the  compass  is  offset  from 
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true  north.  The  errors  generally  associated  with  Gyro  Error  are:  speed 
error,  tangent  latitude  error,  ballistic  deflection  error,  ballistic  damping 
error,  quadrantal  error,  and  gimballing  error. 

Reference  2  contains  definitions  and  discussion  of 
these  error  types.  For  purposes  of  modeling,  they  are  grouped  under  Bearing 
Error. 

Personal  Error  -  This  error  represents  the  fact  that 
the  overall  mean  of  an  individual's  observations  may  tend  to  be  slightly 
larger  or  smaller  than  the  true  measurement. 

Repeater  Offset  Error  -  This  occurs  when  gyrocompass 
repeaters,  which  are  located  at  a  convenient  location  for  measurement,  do  not 
present  the  same  values  as  indicated  on  the  master  gyrocompass. 

Landmark  Misplaced  Error  -  Because  of  the  relative 
weights  of  bearing  measurements  to  sextant  measurements,  the  functional  form 
for  this  error  is  used  only  in  sextant  measurement  simulation. 

2. 3. 1.7  LORAN  Error 

There  are  many  possible  sources  for  error  in  posi¬ 
tion  location  using  LORAN  (reference  3).  Most  do  not  have  a  simple  direct 
relationship  to  the  elements  of  the  L  matrix  of  the  adjustment  equation. 

Model  requirements,  however,  did  not  specify  a  need  for  the  exact  formulation 
of  each  relationship.  All  errors  in  this  group  are  simulated  by  direct 
addition  to  the  measured  LORAN  time  difference  measurement.  All  receiver 
electronic  misadjustments  which  could  cause  measurement  error  are  represented 
by  Electronic  Bias  for  modeling  purposes.  The  other  group  of  error  sources 
which  could  result  in  position  error  are  those  due  to  the  differences  in  the 
predicted  and  actual  transmission  path  of  the  LORAN  signal.  Any  error  source 
in  this  category  is  represented  by  Transmission  Error.  There  is  also  a 
possibility  for  Charting  Error  if  a  chart  is  used  in  positioning  with  LORAN. 

2.3.2  Modeled  Random  Errors 

Table  2-2  provides  a  list  of  the  random  error  sources  subject 
to  study  through  use  of  the  ESM.  The  table  also  provides  an  arbitrary  maximum 
value  for  each  standard  deviation  (s.d.)  of  each  modeled  error  as  well  as  a 
default  s.d.  for  each  modeled  error.  Lacking  an  input  value  for  s.d.,  the 
model  assumes  the  default  value  for  the  s.d.  The  default  s.d.  values  used  are 
taken  from  references  3,  9,  and  10.  These  arbitrary  values  are  intended  to 
represent  the  relative  goodness  of  the  measuring  instruments  and  are  presently 
being  refined/verified  by  other  ongoing  work.  Unless  otherwise  specified,  the 
default  s.d.'s  associated  with  the  measurement  types  are: 

a.  Sextant  -  5  minutes 

b.  Radar  -  30  meters 

c.  Bearing  -  0.5  degree 

d.  LORAN  -  0.1  jjsec 
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TABLE  2-2 


Common  landmark  for  several  observations. 

No  common  landmark  to  any  two  observations. 

The  number  of  measurements  in  each  simulated  geometry  subjected  to  the  error. 
This  quantity  was  arbitrarily  chosen  for  this  study. 


In  some  cases,  it  is  more  appropriate  to  assign  smaller 
standard  deviations  to  chosen  measurement  types  in  order  to  see  the  effect  of 
a  projected  improvement  in  the  repeatability  of  an  instrument. 

The  importance  of  random  error  assignment  cannot  be  over¬ 
emphasized  as  the  effect  of  any  particular  systematic  error  is  dependent  upon 
the  weight  assigned  to  the  observation.  This  is  not  a  serious  constraint 
because,  as  more  refined  measures  of  instrument  precision  are  provided,  the 
model  can  be  modified  and  exercised  accordingly. 

All  but  Landmark  Definition  have  a  simple  inverse  square 
relationship  to  the  diagonal  elements  of  the  weighting  matrix,  W.  The 
relationship  between  landmark  definition  and  the  variance  of  the  resulting 
measurement  is  provided  in  appendix  A. 

The  list  of  random  error  sources  may  not  be  entirely  complete 
but  it  is  considered  that  any  exclusions  are  of  minor  importance  since  omitted 
random  errors  may  be  grouped  with  one  of  the  errors  listed  for  modeling. 

2. 3. 2.1  Sextant/Observer  Standard  Deviation 

The  inherent  stochastic  nature  of  sextant  observa¬ 
tions  is  modeled  through  use  of  this  error.  It  combines  random  error  due  to 
Observer  Standard  Deviation  and  Sextant  Standard  Deviation.  Observer  s.d. 
represents  the  fact  that  even  with  a  perfect  sextant,  an  observer  will  not 
make  the  same  observations  with  repeated  measurements. 

Sextant  Standard  Deviation  is  similar  to  Observer 
Standard  Deviation  except  that  the  observer  is  considered  perfect  while  the 
sextant  is  the  random  error  source.  Both  errors  have  identical  effect  on 
position  error. 

2. 3. 2. 2  Landmark  Definition 

A  landmark  observed  while  making  resection  measure¬ 
ments  may  not  provide  a  clear  and  distinct  image.  This  would  cause  even  the 
perfect  observer  with  the  perfect  sextant  to  have  poor  repeatability  of  a 
measurement.  Some  landmarks  provide  a  very  clear  point  for  alignment  while 
other  landmarks,  such  as  a  wide  water  tower  at  close  range,  do  not.  These 
errors  are  represented  by  Landmark  Definition.  Any  doubt  in  the  quality  of 
horizontal  control  can  be  modeled  also  through  use  of  this  error  (for  doubt 
associated  only  with  the  direction  perpendicular  to  the  line  of  sight  to  the 
landmark). 

2. 3. 2. 3  Radar,  Bearing,  and  LORAN  Standard  Deviation 

The  precision  of  each  radar,  gyrocompass,  and  LORAN 
measurement  is  dependent  on  the  observer  and  the  equipment.  Electronic  noise. 
Inherent  mechanical  fluctuations,  and  visibility  conditions  are  sources  of 
random  error  In  each  measurement.  The  random  errors  are  modeled  directly  by 
user  assignment  of  s.d.'s  to  measurements  simulated.  Since  known  values  of 
s.d.'s  are  not  yet  available,  the  s.d.'s  assigned  for  this  report  are 
estimates. 


2.3.3  Modeled  Outliers 


A  project  requirement  for  the  ESM  is  to  Introduce  a  high 
confidence  outlier  detection  technique.  A  mathematical  description  of  an 
outlier  detection  scheme,  which  can  be  investigated  using  the  ESM,  is  provided 
In  appendix  F,  and  demonstrated  in  section  5.5  In  this  report.  The  technique 
is  fashioned  after  the  "Xz  Goodness  of  Fit"  statistical  test  used  in  the 
physical  sciences.  In  this  test  the  question  is  asked,  "Does  the  observation 
deviate  from  the  functional  model  by  such  a  magnitude  that  It  Is  unlikely  due 
to  random  error  alone?"  The  existence  of  a  large  blunder,  or  of  a  systematic 
error  as  a  cause  of  the  outlier,  can  be  suspected  using  this  method  when  a 
reasonable  estimate  of  the  measurement  s.d.  is  known.  However,  small  blunders 
or  systematic  errors  will  go  unsuspected. 

Examples  of  the  most  common  mistakes  are  applying  Index 
Correction  Improperly  or  of  sighting  a  landmark  which  Is  not  the  one 
selected.  Blunders  are  modeled  similar  to  systematic  errors. 

Table  2-3  provides  a  list  of  often-discussed  blunders  (not 
all  possible  blunders)  and  the  systematic  error  group  in  which  they  are 
assigned. 

Because  the  least  squares  adjustment  equation  uses  linear 
first-order  approximations  to  a  more  corplex  system,  large  blunders  cause 
higher  order  effects  and  cannot  be  modeled  accurately. 


BLUNDER 


Index  Correction  Applied  Wrong 
Sextant  Read  Wrong 
Wrong  Landmark  Observed 
Data  Entry  Mistake 

Incorrect  Radar  Reading 
Incorrect  Gyro  Reading 
Incorrect  LOR AN  Reading 

Angles  Switched 

Uncompensated -for  Systematic  Errors 


TABLE  2-3 
BLUNDERS 

SYSTEMATIC  MODELED  ERROR  GROUP 

Index  Error 
Index  Error 
Landmark  Misplacement 
Index  Error 

Radar  Error 
Bearing  Error 
LORAN  Error 


Too  drastic  to  model 
All  errors  discussed  In  2.3.1* 

*Source  of  blunder  need  not  be  Coast  Guardsmen  (e.g.,  survey  team). 


3.0  GEOMETRY 


The  geometry  of  a  fix  consists  of  ail  the  L0P‘s  which  represent  the 
measurements  that  compose  the  fix,  their  respective  measurement  standard 
deviations,  and  their  orientation  with  respect  to  each  other.  The  “fix 
geometry"  determines  the  strength  of  the  fix.  Effects  of  measurement  errors 
and  computational  mistakes  on  position  error  depend  on  the  strength  of  the 
fix,  which  Implies  that  choice  of  the  "fix  geometry"  is  important  part  to  ESM 
efforts.  The  pitfalls  of  assuming  only  a  few  geometry  layouts  become  evident 
when  considering  the  basic  tenet  "that  all  fixes  are  unique."  Too  much 
information  (studying  many  selected  "fix  geometries")  can  be  as  harmful  as  too 
little  if  useful  Information  is  not  apparent.  The  ESM  model  was  created  with 
two  alternative  approaches  to  the  geometry  problem.  In  the  first  alternative, 
called  the  "Fixed  Geometry"  alternative,  representative  positioning  scenarios 
are  selected  for  study.  The  three  scenarios  selected  for  this  study  are 
described  in  detail  in  the  next  section.  Users  of  this  model  for  other 
studies  are  cautioned  that  if  too  few  geometry  scenarios  are  considered, 
erroneous  conclusions  could  be  drawn  and  important  results  bypassed.  The 
second  alternative,  called  the  "Random  Geometry"  alternative,  generates 
information  from  many  "fix  geometries"  without  creating  so  much  information 
that  it  is  unmanageable.  The  "Random  Geometry"  alternative  is  described 
verbally  in  section  5.2  and  expressed  mathematically  in  appendices  A  and  E. 

3.1  Fixed  Geometry 


Although  many  scenarios  can  be  analyzed  using  the  ESM,  only 
scenarios  called  Harbor,  Near-Coast,  and  Offshore  were  selected  for  study  in 
this  report. 


3.1.1  Harbor 


The  Harbor  scenario  is  composed  of  sextant  measurements  using 
landmarks  that  are  relatively  close  (4000  meters)  to  the  designated  position, 
and  which  surround  the  position  with  equal  angular  spacing  (90°).  Figure 
3-1  depicts  the  geometry  of  the  Harbor  scenario.  This  scenario  was  chosen 
because  many  aids  serviced  by  the  Coast  Guard  are  within  rivers  and  harbors. 

3.1.2  Near-Coast 


This  scenario  is  composed  primarily  of  sextant  measurements 
using  landmarks  equally  spaced  on  a  nearby  (4000  meters)  coastline.  A  fourth 
line  of  position  is  included  using  an  additional  gyrocompass  measurement 
thereby  further  "checking"  the  fix.  The  effect  of  this  additional  "check"  Is 
studied.  Figure  3-2  provides  the  fix  geometry  for  the  Near-Coast  scenario. 

3.1.3  Offshore 


This  scenario  represents  cases  when  sufficient  landmarks  are 
not  available  or  visible  for  resection,  and  is  composed  of  measurements  made 
using  combinations  of  Instruments  other  than  the  sextant.  Comparison  between 
measurement  types  is  made  using  a  symmetrical  geometry  for  each  and  then 
studying  the  respective  position  error.  Figure  3-3  describes  the  Offshore 
geometries . 


R  *  4000  meters 
A  *  90  degrees 

Three  lines  of  position  from  sextant. 
No  other  measurements. 

FIGURE  3-1 

HARBOR  GEOMETRY 


R  *  4000  meters 
A  *  30  degrees 
B  »  60  degrees 

Three  lines  of  position  from  sextant. 
One  line  of  position  from  gyrocompass, 
sighting  on  second  landnark  from  left. 

FIGURE  3-2 

NEAR-COAST  GEOMETRY 


R  ■  4000  meters 
A  »  60  degrees 

Three  lines  of  position  from  radar, 
gyrocompass,  or  LORAN  (Gradient  taken  to 
be  300  meters//jsec). 

FIGURE  3-3 

OFFSHORE  GEOMETRY 
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3.1.4  Other  fix  geometries 


The  operating  instructions  for  using  the  ESM  to  study  other 
scenarios  of  interest  are  contained  in  appendix  B. 

3.2  Random  Geometry 

It  is  seldom  that  all  elements  of  a  large  population  can  be 
studied.  A  common  method  of  studying  large  populations  is  to  draw  a  sample 
from  the  population,  perform  tests  on  it,  and  infer  the  statistics  apply  to 
the  entire  population. 

A  sample  of  "fix  geometries"  using  the  sextant  was  drawn  from  the 
SANDS  (reference  3)  forms  of  USCGC  REDWOOD  (WLM  685),  and  sample  statistics 
were  calculated.  The  direct  route  of  subjecting  each  "fix  geometry"  to 
analysis  proved  to  be  cumbersome,  therefore,  a  Monte  Carlo  routine  was  created 
using  the  sample  statistics.  The  mathematical  equations  used  in  the  Monte 
Carlo  simulation  and  the  sample  statistics  are  provided  in  appendix  E. 

The  "random  geometries"  generated  by  the  Monte  Carlo  routine  were 
subjected  to  error  analysis  and  selected  frequency  distributions  of  error 
effects  were  determined.  The  inference  was  made  that  the  frequency  distribu¬ 
tions  created  represent  the  results  obtainable  by  study  of  all  "real"  aid 
positioning  geometries.  The  critical  reader  will  question  the  validity  of 
representing  all  possible  fix  geometries  by  a  sample  from  the  SANDS  data  of 
only  one  unit.  The  response  to  this  valid  criticism  is  that  the  requirements 
placed  on  the  ESM  efforts  reported  on  herein  calls  for  a  tool  to  be  used  in 
judging  relative  error  effects  among  some  representative  geometries.  If  a 
different  set  of  representative  geometries  (such  as  one  drawn  randomly  nation¬ 
wide)  is  desired,  the  statistics  of  the  sample  can  be  entered  into  the  model 
for  further  study.  The  results  contained  herein  must  be  weighted  by  the  fact 
that  they  are  created  through  use  of  the  USCGC  REDWOOD  sample. 

The  sample  drawn  from  USCGC  REDWOOD  had  too  few  "fix  geome¬ 
tries"  composed  of  radar,  gyrocompass,  and  LORAN  measurements.  Because  of 
this  data  deficiency,  other  modeling  techniques  were  employed.  A  Monte  Carlo 
simulation  of  all  measurements  is  used  adding  reasonable  constraints  upon 
landmarks  sighted  and  crossing  angles.  The  simulated  landmarks  used  for  range 
and  bearing  measurements  were  generated  using  statistics  of  the  REDWOOD 
sample.  However,  successive  landmark  bearings  were  distributed  uniformally 
between  30  and  90  degrees.  LORAN  LOP's  were  simulated  with  similar  crossing 
angles.  Though  the  distributions  used  In  this  work  are  necessarily  biased  due 
to  sampling  from  a  single  unit,  the  point  that  this  error  evaluation  technique 
is  a  powerful  one  is  effectively  made.  More  precise  distributions  can  be 
substituted  if  a  substantial  return  is  indicated. 


4.0  COMPUTER  ADAPTATION  OF  THE  MODEL 


A  set  of  landmarks,  measurements,  and/or  signals  are  chosen  which  define 
a  "fix  geometry"  that  satisfies  the  functional  model  introduced  in  section 
2.0.  The  measurements  are  perturbed  by  subjecting  them  to  an  iteratively  in¬ 
creasing  modeled  error  from  among  those  defined  in  section  2.3.  The  perturbed 
measurements  are  then  transformed  via  the  least  squares  adjustment  equation 
into  a  two-dimensional  location  with  an  associated  two-dimensional  uncertain¬ 
ty.  Various  measures  of  position  error  are  calculated  following  each 
iteration. 

As  inputs  are  required,  the  computer  displays  a  message  to  the  operator. 
The  routine  does  not  have  a  data  entry  error  feedback  system  which  means  the 
operating  instructions  of  appendix  B  should  be  followed  strictly.  The  time 
required  to  study  the  effect  of  one  error  source  on  a  specified  "fix  geometry" 
is  a  few  minutes,  including  data  entry  time.  The  time  required  for  a  "random 
geometry"  case  increases  linearly  with  the  number  of  geometries  desired  in  the 
sample. 

4.1  Required  Inputs 


4.1.1  Fixed  Geometry 

Table  4-1  lists  the  inputs  required  by  the  routine  in  study¬ 
ing  a  "fix  geometry"  (refer  to  appendix  B  for  proper  data  entry  procedures). 

4.1.2  Random  Geometry 


Table  4-2  lists  the  inputs  required  by  the  routine  in  study¬ 
ing  "random  geometries"  (refer  to  appendix  B  for  proper  data  entry  procedures). 

4.2  Outputs  of  the  Error  Sensitivity  Model 

The  outputs  differ  significantly  between  the  two  geometry  approaches. 
In  all  "fixed  geometry"  cases,  the  chosen  error  effect  parameter  is  plotted  as 
a  function  of  the  modeled  error,  whereas  the  "random  geometry"  outputs  are 
frequency  distributions  or  cumulative  frequency  distributions  of  the  selected 
error  effect  parameter  as  a  function  of  five  values  of  the  modeled  error  all 
equally  spaced  within  the  modeled  error  domain  (see  section  5.2).  In  either 
method  the  same  modeled  errors  and  error  effect  parameters  are  available.  The 
modeled  errors  were  discussed  in  section  2.3  and  the  error  effect  parameters 
are  described  in  the  following  subsections.  Refer  to  Results  (section  5.0) 
for  examples  of  the  output  of  the  model. 

4.2.1  AP-to-MPP 


The  point  at  which  all  lines  of  position  cross  in  an  unper¬ 
turbed  "fix  geometry"  is  called  the  assigned  or  assumed  (designated)  position, 
AP.  The  adjusted  set  of  lines  of  position  cross  at  a  point  called  the  most 
probable  position,  MPP.  The  vector  on  the  two-dimensional  positioning  surface 
between  these  two  points  Is  an  error  effect  parameter  called  the  AP-to-MPP 
vector.  The  AP-to-MPP  vector,  following  any  disturbance  of  the  initial  "^ix 
geometry,"  requires  magnitude  and  direction  to  define  it  completely.  The 
direction  is  not  of  value  in  meeting  present  project  requirements.  The  routine 


TABLE  4-1 


FIXED  GEOMETRY  -  INPUTS 

(1)  Number  of  sextant  angular  measurements 

(2)  Number  of  gyrocompass  bearing  measurements 

(3)  Number  of  radar  range  measurements 

(4)  Number  of  LORAN  time  difference  measurements 

(5)  Number  of  landnarks  that  are  common  to  sextant  measurements  (see 
section  5.6,  Limitations  of  the  Model) 

(6)  Standard  Deviations  of  simulated  sextant  measurements  (Default  =  5 
minutes)  (Reference  9) 

(7)  Standard  Deviations  of  simulated  gyrocompass  measurements 
(Default  =0.5  degrees)  (Reference  9) 

(8)  Standard  Deviations  of  simulated  radar  measurements  (Default  =  30 
yards)  (Reference  10) 

(9)  Standard  Deviations  of  simulated  LORAN  measurements  (Default  =0.1 
usee)  (Reference  3) 

(10)  Error  Effect  Parameter  Choice  (see  section  4.2) 

(11)  Error  to  be  studied  (see  section  2.3) 

(12)  Radius  of  target  circle  (optional) 

(13)  Maximum  of  error  to  be  studied 

(14)  Range  and  true  bearing  to  all  landmarks  used  in  sextant  angular 
measurements 

(15)  Range  and  true  bearing  to  all  lanctnarks  used  for  gyro  bearing  measure¬ 
ments 

(16)  Range  and  true  bearing  to  all  landmarks  used  for  radar  range 
measurements 

(17)  Gradient  magnitude  and  direction  for  all  LORAN  time  difference 
measurements 


TABLE  4-2 


RANDOM  GEOMETRY  -  INPUTS 

(1)  Number  of  sextant  angular  measurements 

(2)  Nunber  of  gyrocompass  bearing  measurements 

(3)  Number  of  radar  range  measurements 

(4)  Number  of  LORAN  time  difference  measurements 

(5)  Number  of  landmarks  that  are  common  to  sextant  measurements  (see 
section  5.6,  Limitations  of  the  Model) 

(6)  Standard  Deviations  of  simulated  sextant  measurements  (Default  *  5 
mi nutes ) 

(7)  Standard  Deviations  of  simulated  gyrocompass  measurements 
(Default  -  0.5  degrees) 

(8)  Standard  Deviations  of  simulated  radar  measurements  (Default  =  30  yards) 

(9)  Standard  Deviations  of  simulated  LORAN  measurements  (Default  =  0.1  jjsec) 

(10)  Error  Effect  Parameter  Choice  (see  section  4.2) 

(11)  Error  to  be  studied  (see  section  2.3) 

(12)  Radius  of  target  circle  (optional) 

(13)  Maximum  of  error  to  be  studied 

*(14)  Nunber  of  "fix  geometries"  desired  in  sample 
(15)  Seed  for  random  number  generator  (nine-digit  fraction  between  0  and  1) 

*  To  change  the  parameters  used  to  randomly  generate  sample  geometries,  the 
program  needs  slight  modifications.  Appendix  A  describes  the  program 
listing  that  requires  change. 
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is  not  written  to  provide  the  direction,  but  a  modification  is  not  difficult 
if  studies  of  this  nature  are  desired.  The  mathematical  expression  for 
AP-to-MPP  is  provided  in  appendix  A. 

A. 2.2  2-drms 


The  mathematical  model  results  in  an  MPP  and  in  the  two- 
dimensional  uncertainty  in  the  MPP  for  each  iteration  through  the  modeled 
error.  The  two-dimensional  uncertainty  of  the  MPP  is  defined  with  respect  to 
a  rotated  reference  system  centered  on  the  MPP.  The  angle  between  the  origi¬ 
nal  reference  axis  and  the  new  reference  system  is  determined  such  that  the 
coordinates  are  uncorrelated  in  the  new  system.  The  variance  associated  with 
the  MPP  in  each  of  the  two  orthogonal  directions  defines  completely  the 
uncertainty  in  its  location  in  the  form  of  a  bivariate  normal  probability 
distribution.  Often,  the  two-dimensional  uncertainty  can  be  approximated  by 
one  number.  One  such  number,  2-drms,  is  twice  the  square  root  of  the  sum  of 
the  variances  in  the  orthogonal  reference  system.  It  is  used  to  define  the 
radius  of  a  circle  which  contains  at  least  95  percent  of  the  probability  mass 
of  the  bivariate  normal  distribution. 

4.2.3  Maj*Min  (90S) 

The  contours  of  equal  probability  density  of  the  bivariate 
normal  distribution  form  ellipses.  The  area  of  an  ellipse  which  contains  90 
percent  of  the  probability  mass  of  the  distribution  is  an  error  effect  parame¬ 
ter,  the  Maj*Min  (90%). 

4.2.4  Maj  (90%) 

The  major  semi-axis  of  the  ellipse  described  in  section  4.2.3 
is  an  error  effect  parameter,  the  Maj  (90%) . 

4.2.5  P-in-R 


The  probability  mass  which  is  contained  by  a  circle  of  radius 
R  centered  about  the  assigned  position  is  called  the  P-in-R.  Of  course, 

P-in-R  is  always  less  than  one.  The  probability  is  cal'cul ated  by  a  two-dimen¬ 
sional  integration  of  the  bivariate  probability  distribution  over  the  area  of 
the  designated  circle.  Details  of  this  calculation  are  contained  within 
appendix  C. 


The  value  of  this  error  effect  parameter  can  be  realized  by 
considering  that  it  is  a  function  of  both  random  and  systematic  errors.  A 
drawback  to  its  usefulness  is  that  the  computations  necessary  to  arrive  at  it 
are  very  time  consuming. 

4.2.6  Sum  Sqd  Res 

The  Sum  Squared  Residuals  (Sum  Sqd  Res)  is  an  error  effect 
measure  which  indicates  how  far,  in  terms  of  multiples  of  assumed  measurement 
standard  deviation  units,  the  lines  of  position  of  the  perturbed  system  are 
from  satisfying  the  functional  model.  A  residual  is  the  difference  between  a 
measurement  of  the  perturbed  "fix  geometry"  and  the  respective  measurement  of 
the  adjusted  "fix  geometry."  The  residual  of  each  measurement  is  squared  then 
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divided  by  the  variance  of  that  measurement.  The  weighted  quantities  are 
summed  to  obtain  the  final  parameter.  The  value  of  this  error  effect  parame¬ 
ter  is  demonstrated  in  section  5.5,  Outlier  Detection  and  Identification.  The 
mathematics  associated  with  its  calculation  and  use  are  expressed  in 
appendices  A  and  F. 

4.2.7  Min/Maj 

The  ratio  of  the  minor  and  major  semi-axis  of  the  ellipse 
defined  in  section  4.2.3  is  called  the  Min/Maj  error  effect  parameter. 
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5.0  APPLICATION  OF  THE  MODEL  -  RESULTS 


The  preliminary  considerations  of  Section  2.1  should  be  reviewed  before 
the  reader  becomes  involved  in  studying  the  results  presented  in  this  section. 

A  failure  to  appreciate  the  constraints  of  using  specific  geometries  can 
easily  lead  to  erroneous  conclusions.  On  the  other  hand,  to  interpret  that 
the  expected  values  of  error  effect  derived  from  the  "random  geometry"  method 
are  all  inclusive,  can  just  as  easily  lead  to  erroneous  conclusions.  It  is  a 
combination  of  both  approaches  that  will  prove  useful  in  a  general  analysis  of 
the  positioning  problem. 

The  primary  result  of  this  effort  is  that  an  Error  Sensitivity  Model  has 
been  created  complete  with  operating  Instructions  and  example  applications. 

The  examples  given  do  not  represent  the  total  of  all  information  derivable  by 
use  of  the  model  and  should  only  be  examined  with  regard  to  the  "fix  geome¬ 
tries"  from  which  they  were  derived. 

The  key  for  figure  numbers  throughout  this  section  is: 

X(L)*^^  X=number 

^  Figure  a,  b,  or  c  L=Letter 

Page  of  LL  Data 

Example:  5-OS-2(a)  is  figure  (a)  on 

page  2  of  the  off¬ 
shore  data  in  section 
5  of  the  report. 


Report  Section* 


X-LL- 


Geometry  Type 
H=Harbor  “ 
NC=Near-Coast 
0S=0ff shore 
R "Random 
B=Blunder 


5.1  Fixed  Geometry 

The  results  of  error  analysis  on  the  fixed  geometry  scenarios 
described  in  section  3.1  are  displayed  in  figures  5-H-l(a)  through  5-0S-4(c). 
Refer  to  section  2.3  for  a  description  of  the  modeled  errors  and  section  4.2 
for  a  description  of  the  error  effect  parameters. 

Equivalent  modeled  error  is  introduced  to  allow  comparison  of  the 
effects  of  modeled  errors  pertinent  to  each  measuring  Instrument.  The  equiva¬ 
lent  modeled  error  is  the  error  value  which  causes  100  percent  degradation  of 
the  error  effect  parameter  studied  (i-e.,  either  halves  or  doubles  the  error 
effect  parameter  -  as  appropriate  -  with  respect  to  its  value  when  the  modeled 
error  Is  zero.).  Table  5-1  lists  the  equivalent  modeled  errors  for  the  P-in-R 
error  effect  parameter.  The  equivalent  modeled  errors  are  grouped  by  their 
respective  measuring  Instrument  within  the  columns  and  by  geometry  type  within 
the  rows.  The  table  is  useful  for  comparing  the  different  geometry  scenarios 
and  in  finding  the  relative  importance  of  each  modeled  error  within  each 
"fixed  geometry"  scenario.  Examination  of  table  5-1  and  figures  5-H-l (a) 
through  5-0S-4(c),  pages  33-54,  reveals  information  as  follows: 


Pages  33-50 

The  modeled  errors  have  a  more  significant  increased  effect  on  the  error 
effect  parameters  using  the  Near-Coast  Geometry  than  they  do  in  the 
Harbor  Geometry.  Equivalent  modeled  errors  differ  by  as  much  as  a  factor 
of  two  in  the  case  studied. 
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TABLE  5-1 


FIXED  GEOMETRY  EQUIVALENT  MODELED  ERRORS 


N/A  =  Not  applicable  NWR  =  Not  within  range  NE  *  No  effect 


ERROR# 

HARBOR 

NEAR-COAST 

OFFSHORE 

Sextant 

1-IE 

58  minutes 

25  minutes 

N/A 

1-NCLMM 

70  meters 

50  meters 

N/A 

2-CLMM 

32  meters 

19  meters 

N/A 

1-NCIA 

NE 

NWR 

N/A 

2-CIA 

NE 

NWR 

N/A 

1-OC 

NWR 

NWR 

N/A 

+*3-S/0SD 

8.2  minutes 

7.6  minutes 

N/A 

+*1-NCLMD 

NWR 

NWR 

N/A 

+*2-CLMD 

12  meters 

8  meters 

N/A 

Gyrocompass 

3-BE 

N/A 

NWR 

0.42  degrees 

♦3-BSD 

N/A 

NWR 

0.74  degrees 

Radar 

3-RE 

N/A 

N/A 

24  meters 

♦3-RSD 

N/A 

N/A 

46  meters 

LORAN 

3-LE 

N/A 

N/A 

0.08  usee 

♦3-LSD 

N/A 

N/A 

0.15  usee 

*  The  value  listed 

is  that  which  corresponds  to 

a  P-in-R  which 

is  one-half  of 

that  obtained  using  the  default  standard  deviations  for  the  simulated 
observations. 

+  Target  Circle  5-meter  radius 

#  The  number  represents  the  number  of  measurements  to  which  the  modeled  error 
was  applied. 
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Pages  33-35.  40-42 

The  P-in-R  output  parameter  remains  high  (more  than  0.9)  until  the 
AP-to-MPP  distance  is  about  80  percent  of  the  target  circle  radius. 

Pages  33-96(a),  40-46(a),  51-53(a) 

The  AP-to-MPP  error  effect  parameter  is  a  linear  function  of  systematic 
error . 

Pages *34-35.  38-39.  41-44.  48-49 

Landmark-related  modeled  errors  have  a  more  severe  effect  on  position 
error  when  the  geometry  contains  a  landmark  which  is  common- to  two 
measurements.  In  all  cases  presented,  only  one  common  landmark  was 
simulated.  The  same  effects  would  result  if  two  landmarks,  each  used 
once,  provided  similar  input  errors. 

Pages  36,  43-45 

~J*”  Inclined  Angle  Error  and  Observer  Coincidence  Errors  have,  within  the 

range  investigated,  little  effect  on  the  error  effect  parameters  studied. 

Pages  35,  42,  39(c),  49(c) 

Common  Landmark  Misplacement  and  Common  Landmark  Definition  can  signifi¬ 
cantly  affect  position  error  if  the  error  magnitude  is  of  the  same  order 
as  the  target  circle  radius. 

Pages  37(b),  47(b) 

•  • 

The  Maj  (90X)  error  effect  parameter  is  a  linear  function  of  S/0  Standard 
Deviation. 

Pages  37(a),  47(a) 

The  Maj*Min  (90*)  error  effect  parameter  is  a  quadratic  function  of  S/0 
Standard  Deviation. 

Pages  46(a),  50(c) 

The  simulated  gyrocompass  bearing  measurement  in  the  Near-Coast  geometry 
adds  very  little  to  the  strength  of  the  fix.  Unless  the  standard  devia¬ 
tion  of  gyrocompass  measurements  is  decreased  to  less  than  0.25  degrees, 
it  is  often  of  little  help  in  further  "checking"  a  fix  determined  by 
three-angle  resection  (assuming  a  reasonably  good  geometry  is  used  in  the 
three-angle  fix). 

Pages  51-53 

Given  the  default  measurement  standard  deviations,  three  measurements 
using  the  gyrocompass,  radar,  and  LORAN  receiver  have  a  small  (less  than 
0.1)  chance  of  positioning  the  observer  within  a  target  circle  of  25 
meters  radius,  if  systematic  error  exceeds  0.5  degree,  40  meters,  and  0.1 
psec,  respectively. 

Page  54 

Gyrocompass ,  radar,  and  LORAN  measurements  must  have  standard  deviations 
of  less  than  0.25  degrees,  20  meters,  and  0.05  jjsec,  respectively,  to 
achieve  high  P-in-R  values  (approximately  0.8)  for  a  target  circle  of 
radius  25  meters  with  60°  crossing  angles  (provided  all  systematic 
errors  are  zero). 
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5.2  Random  Geometry 

Cumulative  frequency  distributions  of  error  effect  parameters  are 
displayed  In  figures  5-R-l (a)  through  5-R-6(c),  pages  55-60.  Refer  to  section 
2.3  for  modeled  error  definitions  and  section  4.2  for  error  effect  parameter 
definitions.  Each  figure  displays  five  cumulative  frequency  distributions 
(one  for  each  of  five  different  modeled  error  values)  each  of  which  was 
created  using  the  following  procedure: 

The  error  effect  parameter  is  assigned  a  maximum  value  as  an  option¬ 
al  user  input  (see  appendix  B)  and  the  resulting  range  (0  to  maximum)  is 
divided  into  26  data  accumulation  bins.  Each  geometry  of  the  sample  is 
studied  iteratively,  once  for  each  fifth  of  the  maximum  modeled  error,  with 
the  resulting  error  effect  parameter  tabulated  in  the  appropriate  bin.  If  the 
error  effect  parameter  exceeds  the  specified  maximum,  it  is  tabulated  in  a 
twenty-sixth  bin.  After  all  geometries  of  the  sample  have  been  studied,  the 
sample  fraction  within  each  bin  is  calculated.  The  modeled  error  effect  is 
determined  by  comparing  the  cumulative  frequency  distributions  and  summary 
statistics  which  correspond  to  the  respective  modeled  error  values. 

If  desired,  the  frequency  distribution  can  be  plotted  directly  (not 
cumulative).  The  numbers  which  compose  each  frequency  distribution  on  the 
graph  represent  the  number  of  fifths  of  the  maximum  modeled  error  present  in 
generating  the  respective  frequency  distribution. 

• 

The  expected  value  of  any  error  effect  parameter  is  calculated  from 
the  frequency  distribution  and  is  used  to  rank  order  expected  error  effects. 

At  least  200  randomly  generated  geometries  are  required  to  suffi¬ 
ciently  “smooth”  the  cumulative  distributions,  however,  100  geometries  normal¬ 
ly  provide  enough  data  to  distinguish  among  distributions  on  a  single  graph. 

In  each  case,  the  number  of  sample  geometries,  as  well  as  the  number  of  each 
measurement  type,  are  given  in  the  figure.  In  each  case,  the  number  of  angle 
(A),  bearing  (B),  range  (R),  and  LORAN  (L)  measurements  is  indicated  on  the 
right  side  of  the  graph.  In  addition,  the  target  circle  radius,  R,  is 
provided  in  the  title  block. 

Examl nation  of  figures  5-R-l (a)  through  5-R-6(c),  pages  55  through 
60  respectively,  and  the  expected  values  of  all  error  effect  parameters 
reveals  the  following  expected  trends  and  quantified  information: 


Page  55 

Subjecting  one  sextant  measurement  to  Index  Error  Is  less  severe  than 
subjecting  all  three  measurements  to  Index  Error. 


errors  have  a  more  severe  effect  on  P-ln-R  when  the 
geometry  contains  a  landmark  which  is  common  to  two  measurements  (that 
is,  when  two  measurements  contain  errors). 


Pages  56.  57,  and  59 
Landmark -related 


24 


Pages  55-58 

The  following  errors  cause  at  least  90  percent  of  the  geometries  to  have 
a  P-in-R  of  less  than  0.5.  The  target  circle  radius  is  25  meters. 


ERROR* 

MAGNITUDE 

1-IE 

9U  minutes  (extrapolated  value) 

3-IE 

48  minutes 

1-NCLMM 

100  meters 

2-CLMM 

60  meters 

1-NCIA 

NWR* 

2-CIA 

NWR* 

1-OC 

NWR* 

3-BE 

0.20  degrees 

3-RE 

Less  than  10  meters 

3-LE 

Less  than  0.05  nsec 

f  See  table  2-1  for  list  of  modeled  error  abbreviations  and  codes. 
*  NWR  -  Not  within  range. 

Pages  55-58 

The  following  errors  cause  50  percent  of  the  geometries  to  have  a  P-in-R 
of  less  than  0.9.  The  target  circle  radius  is  25  meters. 


ERROR# 

MAGNITUDE 

nr 

18  minutes 

3-1 E 

12  minutes 

1-NCLMM 

20  meters 

2-CLMM 

18  meters 

1-NCIA 

450  meters 

2-CIA 

300  meters 

1-0C 

16  meters 

3-BE 

0  degrees 

3-RE 

0  meters 

3-LE 

0  psec 

ff  See  table  2-1  for  modeled  error  abbreviations  and  codes. 

Page  57 

Inclined  Angle  Error  and  Observer  Coincidence  Errors  have  very  little 
effect  on  the  P-in-R  under  nearly  all  circumstances  investigated. 

Page  59(a) 

There  is  a  huge  potential  for  increasing  fix  strength  by  reducing  the 
Sextant/Observer  Standard  Deviation.  Fifty  percent  of  the  P-in-R's  are 
moved  from  below  to  above  0.8  by  reducing  Sextant/Observer  Standard 
Deviation  from  5  minutes  to  2  minutes. 
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Landmark  Definition  can  have  a  significant  effect  on  P-in-R  for  many 
geometries  if  its  magnitude  is  greater  than  the  target  circle  radius. 

For  a  5-meter  target  circle,  a  Common  Landmark  Definition  Error  of  10 
meters  causes  P-in-R  to  be  less  than  0.2  for  80  percent  of  the  geometries 
studied.  A  Common  Landmark  Definition  error  of  10  meters  is  unacceptably 
large  by  applicable  survey  standards.  The  average  landmark  is  4000 
meters  from  the  assigned  position  which,  when  Third  Order  Class  II 
Geodetic  Survey  Standards  (reference  19)  are  used.  Indicates  a  possible 
1:5000  or  0.8  meter  uncertainty  in  landmark  location.  Landmark  alignment 
difficulties  or  poor  visibility  may  cause  landmark  definition  errors  as 
large  as  10  meters. 

Page  60 

s.d.'s  of  gyrocompass,  radar,  and  LORAN  measurements  of  greater  than  0.5 
degrees,  30  meters,  and  0.1  jusec,  respectively,  make  positioning  within  a 
25-meter  target  circle  very  Improbable  (approximately  80  percent  of 
geometries  with  P-in-R  of  less  than  0.5  with  no  uncompensated  systematic 
errors  present ) . 

Table  5-2 

Table  5-2  summarizes  the  expected  values  of  the  error  effect  parameters 
as  a  function  of  the  modeled  errors. 

5.3  Rank  Order  of  Modeled  Errors  by  Average  Effect 

There  is  no  conclusive  method  for  rank  ordering  modeled  errors  by 
average  effect.  Any  rank  ordering  for  one  "fixed  geometry"  may  not  apply  to 
any  other  "fixed  geometry."  In  order  to  provide  a  rank  order  with  far-reach¬ 
ing  applicability,  the  modeled  errors  must  be  ranked  with  the  following 
considerations  in  mind: 

a.  On  what  error  source  should  resources  be  expended  to  most 
effectively  decrease  position  error? 

b.  Are  the  modeled  errors  under  consideration  relatively 
negligible? 

c.  What  is  the  frequency  with  which  the  modeled  error  occurs? 

d.  Is  a  suggested  method  of  decreasing  position  error  subject  to 
other  (possibly  more  severe)  error  sources? 

Answers  to  these  questions  are  very  complicated.  Modeled  errors  can 
be  ranked  to  meet  the  project  requirement  by  considering  the  expected  values 
of  the  error  effect  parameters.  The  P-in-R  error  effect  parameter  was  chosen 
to  rank  modeled  errors  by  average  effect. 

Table  5-2  provides  the  information  needed  to  rank  modeled  errors. 
P-in-R  is  given  as  a  function  of  each  modeled  error  over  the  domain  of  the 
modeled  error  selected.  The  function  provided  is  a  least  squares  curve  fit  of 
the  "random  geometry"  derived  P-in-R  averages.  The  correlation  coefficient 
was  very  good  for  nearly  all  error  effect  functions.  In  reviewing  the  func¬ 
tions,  consideration  must  be  given  the  relative  ease  in  effecting  a  decrease 
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In  the  error  throughout  the  aids  to  navigation  macrostructure.  For  example, 
large  improvements  in  P-in-R  are  possible  by  decreasing  the  S/0  Standard 
Deviation  (Slope  *  -0.08/mi n  at  S/0  Standard  Deviation  of  5  minutes).  How¬ 
ever,  decreasing  this  standard  deviation  by  1.0  minute  throughout  the  Coast 
Guard  may  be  very  difficult  compared  to  decreasing  other  errors.  The  error 
effect  function  for  P-in-R  is  of  the  form  y*ae°x.  The  coefficient,  b,  in 
the  exponent  of  each  curve  fit  can  be  used  to  define  another  useful  value, 

1/b.  A  modeled  error  of  magnitude  1/b  decreases  the  expected  P-in-R  value  to 
a/e,  where  a  is  determined  by  the  curve  fit  and  e  is  the  natural  base,  2.718. 

A  review  of  table  5-2  allows  selection  of  the  modeled  errors  which  often  do 
not  contribute  significantly  to  position  error.  The  following  is  a  list  of 
the  modeled  errors  of  least  effect  on  position  error: 

a.  Common  and  Non-Common  Inclined  Angles  -  Very  seldom  is  an 
elevation  difference  of  the  range  modeled  large  enough  to 
cause  significant  increase  in  position  error. 

b.  Observer  Coincidence  -  Most  often  P-in-R  is  low  only  when 
observers  are  separated  by  distances  greater  than  those 
physically  possible  on  the  ships  used  to  position  aids. 

c.  Non-Common  Landmark  Definition  -  The  image  of  a  non-common 
landmark  has  to  be  quite  poor  before  any  significant  decrease 
in  P-in-R  is  suffered. 

The  effectiveness  of  any  decrease  in  an  error  from  a  given  source  is 
indicated  by  the  slope  of  the  error  effect  functions.  Table  5-3  lists  the 
first  derivatives  of  the  P-in-R  error  effect  functions  evaluated  at  selected 
modeled  error  magnitudes.  This  provides  a  measure  of  how  effective  a  decrease 
of  one  model  error  unit  is  in  increasing  P-in-R. 

The  list  in  table  5-3  still  does  not  allow  exact  ranking  of  errors 
by  average  effect  because  there  is  no  data  on  the  frequency  of  occurrence  of 
the  modeled  errors.  However,  the  following  statements  are  made  with  reference 
to  tables  5-2  and  5-3: 

a.  Index  Error,  whether  it  occurs  on  one  or  all  of  the  sextant 
measurements,  has  a  significant  effect  on  P-in-R  and  should 
be  compensated  for  in  a  regular  fashion. 

b.  Landmark  Misplacement  Error  contributes  very  little  to 
position  error  for  displacements  acceptable  to  geodetic 
survey  standards.  However,  unsurveyed  landmarks  can  easily 
cause  a  large  position  error.  Blunders  caused  in  using 
landmarks  differing  from  those  selected  are  obvious  violators 
of  fix  strength. 

c.  Inclined  angles  normally  degrade  position  error  very  little. 

d.  Observer  Coincidence  is  not  significant  enough  to  warrant  any 
more  than  directing  observers  to  stand  as  close  as  possible 
to  one  another. 
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TABLE  5-3 


EFFECTIVENESS  OF  MODELED  ERROR  DECREASE 


MODELED  ERROR  ERROR  EFFECT 

MAGNITUDE  FUNCTION  DERIVATIVE* 

1- IE  *  5  minutes  -0.030exp(-0.025(l-IE)) 

3-IE  3  5  minutes  -0.041exp(-0.039(3-IE)) 

1- NCLMM  3  10  meters  -0.020exp(-0.022(l-NCLMM)) 

2- CLMM  3  10  meters  -0.038exp(-0.034(2-CLMM)) 

1- NCIA  3  100  meters  -0.001exp(-0.001(l-NCIA) ) 

2- CIA  3  100  meters  -0.002exp(-0.002(2-CIA)) 

l-OC  3  5  meters  -0.015exp(-0.015(l-OC)) 

3- BE  3  1.0  degree  -1.52exp(-2.50(3-BE)) 

3-RE  3  50  meters  -0. 049exp(-0. 062 ( 3-RE ) ) 

3-LE  3  0.1  ^sec  -3.05exp(-12.7(3-LE)) 


5-meter  (  3-S/O  SD  3  5  minutes  -0.188exp(-0.164(3-S/0SD)) 

Target  1 

Circle  {  1-NCLMD  3  10  meters  -0.005exp(-0.012(l-NCLMD)) 

Radius  J 

(  2-CLMD  3  10  meters  -0.007exp(-0.023(2-CLMD)) 

25-meter  (  3-BSD  3  0.5  degree  -2.97exp(-2.29(3-BSD)) 

Target  \ 

Circle  (  3-RSD  3  30  meters  -0.038exp(-0.034(3-RSD)) 

Radius  J 

(  3-LS0  3  0.1  *isec  -14.6exp(-U.9(3-LSD)) 


EFFECTIVENESS 

P-in-R/unit 

-0. 026/mi n 

-0. 034/mi n 

-0.016/met 

-0.027/met 

-0.001 /met 

-0.002/met 

-0.014/met 

-0.124/deg 

-0.002 /met 

-0.850/jjsec 

-0.082/mi n 
-0.004 /met 
-0.006 /met 

-0.940/deg 

-0.014/met 

-4.40/jisec 


*  The  coefficient  (1-IE)  in  the  exponent  represents  a  modeled  Index  Error 
applied  to  one  angle  (not,  1  minus  the  Index  Error);  this  is  also  true  for 
other  modeled  errors. 


e.  Improvements  in  Bearing,  Range,  and  LORAN  Errors  of  0.25 
degree,  10  meters,  and  0.05  ysec,  respectively,  increase 
P-in-R  by  less  than  5  percent  at  points  considered  in  table 
5-3. 

f.  A  decrease  in  S/0  Standard  Deviation  can  greatly  increase 
P-in-R. 

g.  Common  Landmark  Definition,  on  the  average,  does  not  affect 
position  error  as  drastically  as  Landmark  Misplacement,  but 
should  not  be  neglected  when  either  large  or  unclear  land¬ 
marks  are  used. 

h.  Decreases  in  Bearing,  Range,  and  LORAN  Standard  Deviations  of 
0.25  degrees,  10  meters,  and  0.05  psec,  respectively,  can  in 
many  cases  increase  P-in-R  by  more  than  0.20.  This  can  make 
them  suitable  for  positioning  aids  when  a  25-meter  target 
circle  is  adequate. 

5.4  Priority  Error  Designation 

The  discussion  in  sections  5.1  through  5.3  leads  to  designation  of 
the  following  modeled  errors  as  “Priority  Average  Errors"  (based  on  assump¬ 
tions  made). 

a.  Index  Error 

b.  Landmark  Misplacement 

c.  S/0  Standard  Oeviation 

d.  Bearing,  Range,  and  LORAN  Standard  Deviation 

5.5  Outlier  Detection  and  Identification 


An  R&D  project  requirement  called  for  formulation  of  a  high  confi¬ 
dence  outlier  detection  technique  for  application  when  aid  positioning.  Two 
mathematical  techniques  are  described  in  appendix  F  and  one  is  discussed  in 
this  section.  Section  2.3.3  provides  a  list  of  some  of  the  blunders  possible 
in  the  positioning  process. 

The  mathematics  in  appendix  F  indicate  that  the  presence  of  an 
outlier  becomes  apparent  in  many  cases  by  performing  a  X2  test  (reference 
8)  on  the  sum  of  the  squared  weighted  residuals.  The  test  is  designed  to 
indicate  when  the  consistency  of  the  lines  of  position  is  so  poor,  compared  to 
what  is  expected,  that  it  probably  was  not  the  result  of  only  random  error. 

An  analogy  to  blunder  detection  is  to  consider  the  outlier  as  a  signal,  and 
random  errors  as  the  noise.  If  the  slgnal-to-noise  ratio  is  large  enough,  the 
X2  test  should  indicate  an  Inconsistency  in  the  measurements  used  to  create 
the  fix;  the  inconsistency  may  be  large  enough  to  be  flagged  as  an  outlier 
containing  a  blunder.  If  the  fix  Is  composed  of  three  lines  of  position, 
further  examination  is  required  to  identify  the  suspicious  observations.  In  a 
fix  containing  four  independent  lines,  all  four  can  be  tested  and  if  found 
excessively  inconsistent  by  failure  of  the  x2  test,  each  of  the  four 
combinations  of  three  can  be  tested  in  turn  against  the  fourth  line  (see 


apppendix  F).  In  cases  where  the  geometry  includes  common  landmarks  (e.g.,  a 
four-line  fix  with  only  four  landmarks,  not  independent  lines  of  position),  a 
blunder  in  using  a  landmark  other  than  the  selected  common  landmark  could 
disturb  two  or  more  lines  of  position  and  the  blunder  identification  procedure 
could  break  down  because  of  this  dependency.  Four  good  landmarks  are  required 
to  find  an  inconsistency  due  to  one  poorly  located  landmark.  Fixes  with  more 
than  four  LOP's  can  be  tested  similarly,  but  the  possible  combinations  grow 
rapidly. 

Figure  5-B-l(a),  Page  61,  shows  frequency  distributions  of  the  Sum 
Sqd  Res  of  a  three-angle  fix  using  the  random  geometry  method;  there  is  one 
distribution  for  each  of  five  Index  Error  (IE)  values  to  which  the  fixes  were 
subjected.  One  angle  was  subjected  to  error  and  the  fixes  were  tested  for 
excessive  inconsistency  at  the  95  percent  confidence  level 
(i.e.,  2  (0.95)*3.8).  The  reference  s.d.  used  was  5  minutes  (appendix  F). 

The  following  information  is  obtained  by  review  of  the  figure: 

a.  Eighty  percent  of  all  fix  geometries  subjected  to  12  minutes 
of  Index  Error  go  unsuspected. 

b.  Forty-five  percent  of  all  fix  geometries  subjected  to  24 
minutes  of  Index  Error  go  unsuspected. 

c.  Thirty-two  percent  of  all  fix  geometries  subjected  to  36 
minutes  of  Index  Error  go  unsuspected. 

d.  Twenty-two  percent  of  all  fix  geometries  subjected  to  48 
minutes  of  Index  Error  go  unsuspected. 

e.  Twenty  percent  of  all  fix  geometries  subjected  to  60  minutes 
of  Index  Error  go  unsuspected. 

If  the  S/0  Standard  Deviation  was  improved,  many  more  blunder 
candidates  could  be  found.  The  ESM  can  be  used  in  the  fixed  geometry  mode  to 
study  the  outlier  rejection  levels  of  any  specified  "fix  geometry."  Figures 
5-8-1 ( b-c)  are  examples  of  such  a  study  on  the  Harbor  geometry,  employing  the 
chosen  5-minute  standard  deviation  of  sextant  measurements.  Figure  5-B-l(b) 
displays  the  effect  Index  Error  has  on  the  Sum  Sqd  Res  when  it  is  subjected  to 
one  measurement  of  the  Harbor  geometry.  The  outlier  detection  scheme  would 
detect  Index  Errors  greater  than  15  minutes.  This  plot  may  be  different  when 
subjecting  the  error  to  other  measurements  in  the  "fix  geometry."  Figure 
5-8-1 (c)  displays  the  effect  Index  Error  has  on  the  Sum  Sqd  Res  when  it  is 
subjected  to  all  measurements  of  the  Harbor  geometry.  The  Inconsistency  of 
the  measurements  becomes  apparent  at  a  much  smaller  value  of  Index  Error 
(7  minutes). 

The  5-mlnute  measurement  standard  deviation  used  can  be  changed  for 
any  study  to  suit  the  requirements  on  outlier  detection.  Likewise,  other 
confidence  levels  can  be  used  In  the  2  tests. 

5.6  Limitations  of  the  Model 

The  following  list  contains  limitations  of  the  Error  Sensitivity 
Model.  The  limitations  remain  because  solutions  were  either  too  time-consum¬ 
ing,  unjustly  complicated,  or  beyond  th^scope  of  this  report. 


a.  Study  of  the  effect  of  combinations  of  errors. 

b.  Study  of  the  effect  of  errors  using  various  combinations  of 
common  and  non-common  landmarks. 

c.  Study  of  geometry  scenarios  using  various  sets  of  default 
standard  deviations. 

d.  The  instability  of  numerical  integration  of  very  peaked 
probability  distributions  (appendix  C)  exists  and  has  not 
been  dealt  with  adequately. 

e.  The  model  has  no  built-in  landmark  selection  algorithm  and 
therefore  should  not  be  considered  a  Landmark  Selection 
Routine. 
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6.0  CONCLUSIONS 


The  following  list  summarizes  results  obtained  by  work  on  the  ESM.  The 
conclusions  are  based  on  assumptions  made  throughout  this  report  and,  there¬ 
fore,  must  not  be  considered  without  knowledge  of  the  assumptions: 

a.  A  computer  routine  has  been  written  which  considers  random 
errors,  systematic  errors,  and  blunders  in  a  mathematical  simulation  of  the 
Coast  Guard  aid  to  navigation  positioning  process. 

b.  Error  ranking  by  average  effect  on  position  error  has  been 
accomplished  (section  5.3).  The  following  error  sources,  as  specified  in 
table  2.1,  have  the  most  significant  effect  on  position  error: 

1.  Index  Error 

2.  Landmark  Misplacement  Error 

3.  Sextant  Observer  Standard  Deviation 

4.  Bearing,  Range,  and  LORAN  Standard  Deviation 

c.  Calculation  of  the  probability  of  sinker  release  within  a 
target  circle  (P-in-R)  is  useful  in  error  analysis  of  the  positioning  process. 

d.  Unless  the  precision  of  bearing,  range,  and  LORAN  measure¬ 
ments  is  significantly  better  than  that  used  in  this  report,  their  use  to 
improve  confidence  of  sextant  fixes  is  of  little  consequence.  However, 
independent  checks  may  be  useful  in  identifying  blunders. 

e.  Landmarks  common  to  more  than  one  measurement,  on  the 
average,  can  cause  larger  position  error  than  landmarks  used  for  only  one 
measurement. 

f.  A  high  confidence  outlier  detection  procedure  seems  tractable. 

g.  The  visual  definition  of  a  landmark  can  contribute  signifi¬ 
cantly  to  position  error  under  certain  circumstances. 

h.  Observer  Coincidence  and  Inclined  Angle  errors  seldom  add 
significantly  to  position  error. 

i .  Gyrocompass ,  radar ,  and  LORAN  measurements  can  be  used  to 
position  with  high  confidence  only  if  their  measurement  s.d.'s  are 
significantly  less  than  those  simulated. 

j.  Landnark  Misplacement  errors  must  exceed  geodetic  survey 
standards  to  have  a  significant  effect  on  position  error. 
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7.0  RECOMMENDATIONS 


The  following  list  results  from  work  done  on  the  Positioning/Error  Model. 

a.  The  ESM  should  be  considered  for  use  in  development  of  stan¬ 
dards  and  requirements  for  aid  positioning. 

b.  The  ESM  should  be  used  to  study  specific  areas  of  concern. 
Examples  are  requirements  on  sextant  quality  and  survey  quality. 

c.  Further  investigation  into  the  value  of  P-in-R  as  a  planning 
tool  should  be  performed.  Numerical  stability  and  time  considerations  should 
be  examined  thoroughly.  The  inverse  of  P-in-R,  that  is,  the  radius  of  a 
circle  required  to  contain  some  desired  probability  mass  (R-for-P),  should  be 
studied  for  possible  usefulness. 

d.  The  outlier  detection  scheme  briefly  introduced  should  be 
thoroughly  examined  for  practical  use  in  field  procedures. 

e.  The  position  error  magnitude  calculated  in  this  report  should 
be  compared  to  those  errors  associated  with  sinker  drop  and  the  watch  circle 
of  the  buoy. 
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appendix  a 


PROGRAM  DESCRIPTION  AND  FLOWCHART 


Program  Description 

The  main  routine  of  the  ESM  is  called  POSER.  This  routine  contains  nearly  all 
the  input/output  function  of  the  model.  POSER  calls  the  subroutine  FIXED  to 
define  a  particular  "fix  geometry"  and  the  subroutine  RANDOM  to  set  up  for 
generation  of  the  sample  "Random  geometries."  The  FIXED  routine- automatically 
calculates  and  stores  the  elements  of  a  parameter  matrix.  The  parameter 
matrix  contains  all  the  information  necessary  to  iteratively  study  any  par¬ 
ticular  fix  geometry.  POSER  calls  DEFAULT  to  establish  the  random  error 
parameters  of  each  line  of  position  within  the  parameter  matrix.  In  the 
Random  Geometry  method,  POSER  calls  GENERATE  each  time  a  new  fix  geometry  is 
needed  for  study.  In  either  method  POSER  calls  GRADIENT  to  calculate  the 
gradient  vector  magnitudes  and  directions  of  each  line  of  position  and  stores 
them  within  the  parameter  matrix.  POSER  then  uses  ERROR  to  branch  to  one  of 
the  error  input  subroutines  which  are  numbered  1  through  18. 

The  error  input  subroutines  complete  the  parameter  matrix  formulation  and 
provide  the  errors  that  perturb  the  system. 

POSER  then  constructs  the  A  matrix  of  equation  2-1  by  calling  PARTIALS.  The  W 
matrix  follows  when  POSER  uses  WEIGHTS.  The  systematic  error  perturbation  is 
then  entered  when  POSER  uses  BIAS.  The  system  is  now  perturbed  and  ready  for 
a  least  squares  adjustment  which  is  performed  through  the  matrix  transforms  of 
equation  2-1  in  REDUCE.  The  completed  transformation  is  then  analyzed  and  the 
error  effect  parameters  are  stored  by  ZOUTPUT.  At  this  point  two  different 
paths  are  possible  depending  on  what  geometry  method  is  being  used.  If  a 
fixed  geometry  is  being  studied,  POSER  calls  TABULATE  to  plot  and  print  the 
selected  error  effect  versus  the  present  value  of  the  modeled  error.  In  the 
random  geometry  method  TABULATE  records  the  result  of  this  particular  fix 
geometry  then  calls  OUTLIERS  which  searches  the  results  for  outliers  and 
computes  statistics  of  the  error  effect  sample.  If  OUTLIERS  finds  a  large 
error  effect  (more  than  two  standard  deviations  different  from  the  mean  of 
previous  values  of  this  parameter),  it  prints  the  parameter  matrix  of  that 
particular  fix  for  future  study.  POSER  then  returns  to  GENERATE  to  continue 
the  Monte  Carlo  process.  Once  all  geometries  have  been  studied  and  the  error 
effects  tabulated,  POSER  calls  PLOTOUT  and  PRINTOUT  to  provide  the 
distributions  to  the  user.  The  user  has  a  paper-tape  printout  of  his  input 
parameters,  a  plot  of  the  results,  and  a  tabulation  of  the  results  at  iterated 
error  values.  In  addition,  the  random  geometry  user  has  a  list  of  outliers  to 
study  for  indications  of  bad  or  good  geometries. 

The  format  used  in  describing  each  subroutine  or  subfunction  in  the  Error 
Sensitivity  Model  Includes: 

a.  A  definition  of  each  variable  used  In  the  equations  of  the  sub¬ 
routine  (unless  they  have  been  defined  previously  in  this  appendix). 

b.  The  equations  used  in  the  subroutine 
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START, 
[POSER  I 


FIGURE  A-l 

ERROR  SENSITIVITY  FLOWCHART 


A-2 


c. 


A  brief  explanation  of  the  purpose  of  the  subroutine  or  subfunction 
including  any  circumstances  or  conditions  important  in  the  develop¬ 
ment  of  the  routine. 


Figure  A-l  is  a  flowchart  of  these  routines  as  they  are  applied  in  the  pro¬ 
grammed  sequence. 

POSER 


Definitions  of  Variables:  N/A 
Equations:  N/A 

Remarks:  POSER  is  the  main  input/output  (I/O)  routine  whose  overall 

purpose  is  space  allocation,  data  storage,  and  program  execu¬ 
tion. 


CHANGE 


Definitions  of  Variables: 


0  is  the  s.d.  storage  array 

d-j  a  s.d.  of  ith  measurement  type 


I  1 

if  i  =2  then  d^ 
3 
'  4 


I  Sextant  measurement  s.d. 
Gyrocompass  measurement  s.d. 
Radar  measurement  s.d. 

LORAN  measurement  s.d. 


Equations:  N/A 

Remarks:  The  ESM  assumes  measurement  s.d.  default  values  for  the 

sextant,  gyrocompass,  radar,  and  LORAN  receiver  of:  5 
minutes,  0.5  degrees,  30  meters,  and  0.1  jjsec,  respectively. 
In  the  case  where  other  default  values  are  desired,  the  user 
instructs  POSER  to  call  CHANGE  to  alter  the  necessary  di 
values. 


ERROR 

Definitions  of  Variables:  ri  -  Number  assigned  to  error  to  be  studied 

1  Index  Error 

2  Non-Common  Landmark  Misplacement 

3  Common  Landnark  Misplacement 

4  Range  Error 

5  Bearing  Error 

6  LORAN  Error 

7  Non-Common  Inclined  Angle 

8  Common  Inclined  Angle 

9  Observer  Coincidence 

10  *S/0  Standard  Deviation 

11  j*S/0  Standard  Deviation 

12  Non-Common  Landmark  Definition 

13  Common  Landmark  Definition 

14  Bearing  Standard  Deviation 

15  Range  Standard  Deviation 

16  LORAN  Standard  Deviation 

17  Vacant 

18  Vacant 
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jp  wmm 


Equations:  N/A 

Remarks:  ERROR  transfers  execution  to  the  subroutine  that  corresponds 

to  the  error  being  studied,  and  returns. 


WEIGHTS 


Definitions  of  Variables:  P  is  the  (n  x  10)  parameter  matrix 

n  is  number  of  lines  of  position 


Pij 
If  Pil 


*  j**1  parameter  of  itfl  line  of  position;  (i  =  1-  to  n) 

II  (  sextant 

2  then  measurement  type  is  gyrocompass 
3  radar 

4  \  LORAN 


If  Pil 


1 

2  then  p^ 

3 

4 


x  coordinate  of  left  landmark  of 
horizontal  angle 

x  coordinate  of  landmark  used  for  bearing 
x  coordinate  of  landmark  used  for  range 
direction  of  LORAN  gradient  vector 


If  Pil 


1 

2  then  p^ 
4 


y  coordinate  of  left  landmark  of 
horizontal  angle 

y  coordinate  of  landmark  used  for  bearing 
y  coordinate  of  landmark  used  for  range 
magnitude  of  LORAN  gradient  vector 


Pi4  3  x  coordinate  of  right  landmark  of  horizontal  angle  (zero 
for  other  measurement  types) 

Pi5  3  y  coordinate  of  right  landmark  of  horizontal  angle  (zero 
for  other  measurement  types) 

Pjg  =  measurement  which  determines  the  ith  nne  jf  position  of 
the  unperturbed  system 

Pi 7  3  measurement  s.d.  of  the  ith  measurement 

p18  3  systematic  error  for  i™  measurement 

Pig  3  gradient  magnitude  of  itn  line  of  position 

Pi io  *  positive  gradient  direction  (that  direction  normal  to  the 

line  of  position  that  will  increase  the  measurement) 


W  is  the  (n  x  n)  weighting  matrix 

Wii  3  weighting  factor  for  ith  line  of  position  (all  off 
diagonal  elements  are  zero) 
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Equations: 


If  Pil 


Remarks: 


1 

/  10800  ' 

\2 

minutes 

1 

1  7 

/  180  ' 

\2 

degrees 

lP'71r/ 

/ 

■  then  Wff  = 

F 

\  <1 

with  Pf7  in 

1  3 

l 

'  f  1 

r 

meters 

W  - 

I 

A 

l  1  ' 

Y 

usees 

*T 

\  Pi  7  , 

1 

The  measurement  s.d.! 

>  are  used 

to  form 

the  weighting  matrix. 

BIAS 


Definitions  of  Variables:  L  is  the  (n  x  1)  observation  vector 

*  systematic  error  perturbation  of  ith 
line  of  position 


Equations: 


1 


if  Pil 


Pi8  7T 
10800 

Pi 8  rr 

Pi8 

Pi8 


Remarks:  The  systematic  error  is  transferred  from  the  parameter  matrix 

to  the  observation  matrix  with  units  conversion. 


REDUCE  (reference  3) 

Definitions  of  Variables:  A  is  the  (n  x  2)  matrix  of  partial  derivatives 


a11 


anif 
d  x 


*12 


3mf 

IT 


mf  ■  variable  which  represents  the  measurement  which 
determines  the  1th line  of  position 


V  is  the  (2x2)  matrix  of  eigenvectors 


t  •  angle  between  x-axis  and  major  semi-axis  of  confidence 
ellipse 

v  [cos  t  -sin  t] 

[sin  t  cos  tj 


A-5 


X 

N 


-(AtWA)"1  atwl. 


Ax 

Ay 


from  A P  to  MPP 


(A^WA)“1  =  (2x2)  covariance  matrix  of  MPP 

(not  normally  diagonal  in  the  North  oriented 
reference  system) 


R  =  AX  +  L  *  (nxl) 
Q  =  VTNV 


0 

0  °y? 


residual  matrix 

rotated  covariance  matrix  of  MPP 
(diagonal) 


Remarks :  The  matrix  operations  in  this  subroutine  form  the  basis  of 

the  Error  Sensitivity  Model.  The  diagonal  elements  of  the  Q 
matrix  are  the  two  variances  of  the  bivariate  probability 
distribution  which  describes  the  position  uncertainty.  The  X 
matrix  contains  the  x  and  y  displacement  of  the  MPP  from  the 
unperturbed  AP.  All  output  parameters  are  computed  from  the 
elements  of  X,  Q,  and  the  residual  matrix,  R. 


ZOUTPUT 


Definitions  of  Variables:  Z  is  the  (1  x  7)  row  vector  of  the  error 

effect  parameters. 

Equations: 

zi  =  AP-to-MPP  =  Vax2  +  Ay2 
Z2  *  2-drms  =  2^  <rx?  +  cry? 
z3  *  Maj*M1n  (9056)  *  (2.15)2  *t'  *  y1 
Z4  *  Maj  (90*)  *  2.15  ^  max  (<rx?j  o-y?) 

Z5  =  P-in-R  =  (see  appendix  C) 

n  - 

Zg  *  Sum  Sqd  Res  *  £  H  w^ 

1*1 


Z7  *  Min/Maj  *^min  (<r2,  <ry2)/max  {<rj,  c r  ?) 


Remarks:  The  output  parameters  are  transferred  from  Q,  X,  R  to  the  error 
effect  parameter  matrix. 

TABULATE 

Definitions  of  Variables:  F  is  the  (6  x  26)  error  effect  parameter  tab¬ 
ulation  matrix 

p  is  a  number  1-7  (Z^  -  Z7,  respectively) 
which  Indicates  the  error  effect  parameter  to 
be  tabulated  for  plotting  and  printout 

f1j  is  the  Jth  error  effect  increment  bin 
for  the  1th  modeled  error  value 

A-6 


Remarks: 


In  the  fixed  geometry  method,  TABULATE  plots  and  prints  the 
calculated  error  effect  parameter  versus  the  modeled  error 
for  each  iteration.  In  the  random  geometry  method,  the  value 
of  the  error  effect  parameter  is  assessed  and  the  count 
incremented  in  the  appropriate  bin  for  later  frequency 
distribution  calculations. 


PLOTOUT 


Definitions  of  Variables:  N/A 
Equations:  N/A 

Remarks :  PLOTOUT  calculates  the  frequency  distributions  and  offers  the 

user  the  option  of  either  a  frequency  distribution  or  cumula¬ 
tive  frequency  distribution  output. 


PRINTOUT 


Definitions  of  Variables:  0  =  6x7x4  error  effect  parameter 

statistics  array 

°ij3  =  mean  error  effect  parameter 

for  the  i*-11  modeled  error  value 


0^4  *  standard  deviation  of  the  jth 

error  effect  parameter  for  the  ith 
modeled  error  value 


Oiji  and  0^2  are  the  sum  and  sum-squared 
values  used  to  caculate  the  means  and  s.d.'s 


Equations:  N/A 

Remarks:  PRINTOUT  lists  the  frequency  distributions  of  each  output 

parameter  and  the  array  of  error  effect  parameter  statistics. 


SN  (reference  13) 

Definitions  of  Variables:  rnd(l)  is  a  uniformly  generated  random  number 

between  0  and  1. 

Equations:  SN  *  ^-2  ln(rnd(l))  cos  (2irrnd(l)) 

Remarks:  SN  is  a  subfunction  which  transforms  a  uniformly  distributed 

random  number  into  a  normally  distributed  random  number  with 
a  mean  of  0  and  a  standard  deviation  of  1.  It  is  used  in  the 
random  geometry  method  for  generating  normally  distributed 
angles,  ranges,  and  LORAN  gradients. 

OIS 


Definitions  of  Variables:  <rx i  ■  standard  deviation  in  x*  direction 


*  standard  deviation  in  y'  direction 


Remarks: 


This  subfunction  defines  the  bivariate  normal  distribution 
which  is  integrated  in  two  dimensions  over  the  target  circle 
(appendix  C)  to  calculate  P-in-R. 

DEFAULT 


Definitions  of  Variables:  N/A 
Equations:  If  p^  =  j  then  dj  *  p^ 

Remarks:  DEFAULT  transfers  the  default  measurement  standard  deviations 

from  the  storage  array  (D)  to  the  active  parameter  matrix  (P) 
before  each  iterative  increase  in  the  modeled  error. 

GENERATE 


Definitions  of  Variables:  M  is  the  (n  x  3  x  2)  array  of  geometry  sample 

set  statistics. 


i  *  ith  line  of  position 


if 


J  -  {2 

(  3 


then 


mijl  = 
mij2  3 


(  mean  of  ranges  to  left  landmarks 
/  mean  of  ranges  to  right  landmarks 
I  mean  of  angles  between  landmarks 

Is.d.  of  ranges  to  left  landmarks 
s.d.  of  ranges  to  right  landmarks 
s.d.  of  angles  between  landmarks 


Equations:  N/A 


Remarks :  GENERATE  calls  subroutines  NA,  NB,  NR,  NC  as  necessary  to 

create  a  new  geometry  for  the  next  iteration  of  the  Monte 
Carlo  method. 


NA 


Definitions  of  Variables: 


SN  Is  the  normally  distributed  random  number 
with  mean  *  0,  s.d.  *  1. 


Equations: 

riL  =  fni  11  +  n»U2  SN  =  range  to  left  landmark  of  ith  measurement 

riR  =  mi 21  +  mi 22  SN  s  range  to  right  landmark  of  itb  measurement 

ai  3  mi 3i  +  mj32  SN  =  itb  angular  measurement 

Remarks :  NA  generates  the  ranges  to  and  angle  between  two  landmarks 

for  a  sextant  measurement.  Additional  logic  and  equations  in 
NA  establish  parameter  matrix  elements  for  the  landmarks 
generated.  See  appendix  E  for  the  sample  statistics  used  in 
the  ESM. 


NB 


Definitions  of  Variables:  bj  is  the  bearing  measurement  for  itb 

line  of  position. 


Equations: 

bi+l“bi  =  6°rnd(l)  +  30  =  U  [30,90] 

Remarks:  This  subroutine  generates  gyrocompass  measurements  with 

crossing  angles  uniformly  distributed  between  30  and  90 
degrees.  The  ranges  to  the  landmarks  being  simulated  are 
generated  as  in  NA. 


Definitions  of  Variables:  r<  is  the  range  measurement  for  the  ith 

line  of  position. 


Equations: 

ri  *  mi xi  +  m-j  12  SN  3  range  measurement  for  the  itb  line  of  position, 
bi+l-bi  *  60rnd(l)  +  30  =  u[30,9o] 

Remarks:  The  ranges  are  generated  as  in  NA  and  the  crossing  angles  the 

same  as  in  NB. 


NL 


Definitions  of  Variables:  a  mean  L0RAN  gradient  in  meters//&sec 

mil2  *  s*d.  °f  LORAN  gradients  in 
meters/^isec 


Equations: 

Pi 3  3  mi ii  +  mji2  SN  a  gradient  magnitude  of  ith  LORAN  line  of 

position 


A-9 


Remarks: 


The  gradient  magnitudes  are  generated  from  a  LORAN  gradient 
distribution  simulation  (appendix  E).  The  crossing  angles  of 
LORAN  line  of  positions  are  generated  uniformly  between  30 
and  90  degrees  as  within  NB  and  NR. 


GRAOIENT 


Definitions  of  Variables:  N/A 


Equations:  N/A 

Remarks:  GRADIENT  is  a  branching  subroutine  which  calls  GA,  GB,  GR,  or 

GL  to  calculate  the  pjg  and  pil0  parameter  matrix 
elements  which  are  the  gradient  magnitude  and  positive 
gradient  direction,  respectively. 


GA 


Definitions  of  Variables:  r6  and  r7  are  dummy  test  variables 

atn  =  arc  tangent 


Equations: 


Pi  9 


(Pil  +  Pi 3 H Pi 4  *  Pi §) 
(Pi2-Pi4)2+  ( Pi 3“Pi 5 ) 2 


1 


P i 3 C P i 4^ if)  ~  Pl5(Pl2+Pl3)  . 


2(pi4Pi3  -  Pi 2Pi5^ 


Pj 2^Pj 4~*~Pj 5)  ~  Pj4(Pi2'HPi3)  .  7 
^(Pi2Pi5  "  -P-i 4P i  3 


0.00029  =  gradient  of  ith  line 
of  position 


=  r6 


=0  and  r7  >  0 
=0  and  r7  <  0 

if  r6(  then  p^ig  = 

>0 


<0 


0 

180 

90-atn(r7/r6) 

270-atn(r7/r6) 


Remarks: 


GA  calculates  the  gradient,  pi9,  and  the  positive  gradient 
direction,  png,  for  each  line  of  position  determined  by 
resection  ana  places  them  in  the  active  parameter  matrix  for 
later  use. 


GB 


Definitions  of  Variables:  N/A 


Equations: 

2  2  ^ 

(Pi2+Pi3)?  0.017453  =  pig 

Pi  10  =  Pi 6  -  90  (but  less  than  360  degrees) 

Remarks:  G8  calculates  the  gradient  and  positive  gradient  direction 

for  all  bearing  measurements. 


GR 


Definitions  of  Variables:  N/A 


Equations: 


Pi  9  =  1 
if  pi2  = 


/0  and  Pi  3 
10  and  Pi 3 


r 

(>0 


>  0 
<  0 


then  Pi io  = 


180 

0 

90  -  atn(pi3/pi2) 
270  -  atn(pi3/Pi2) 


Remarks:  GR  calculates  the  gradient  and  positive  gradient  direction 

for  all  range  measurements. 


GL 


Definitions  of  Variables:  N/A 
Equations:  N/A 

Remarks:  GL  transfers  the  input  gradient  and  positive  gradient  direc¬ 

tion  of  L0RAN  measurements  from  Pi 2  and  Pi 3  t0  Pig  and 
Pi 10’  respectively. 

PARTIALS 


Definitions  of  Variables:  N/A 
Equations:  N/A 

Remarks:  PARTIALS  uses  the  equat’  “»s  derived  in  appendix  G  of  refer¬ 

ence  3  to  establish  all  cements  of  the  A  matrix.  It  calls 
an  AA,  AB,  AR,  and  AL  for  angles,  bearings,  ranges,  and  L0RAN 
measurements,  respectively. 

OUTLIERS 


Definitions  of  Variables:  N/A 
Equations:  N/A 
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Remarks: 


OUTLIERS  calculates  the  mean  and  s.d.  of  the  random  geometry 
error  effect  parameters  for  each  increment  in  the  modeled 
error.  If  (after  30  geometries  have  been  sampled)  any  error 
effect  parameter  exceeds  two  standard  deviations  in  either 
direction  from  the  mean,  the  BAD  or  the  GOOD  subroutines  are 
called  to  print  the  respective  parameter  matrix  for 
additional  examination. 


PROBABILITY 

Definitions  of  Variables:  N/A 


Equations:  N/A 

Remarks :  PROBABILITY  is  discussed  in  appendix  C.  PROBABILITY  calcu¬ 

lates  the  P-in-R  error  effect  parameter. 

1^  (Index  Error) 


Definitions  of  Variables:  d  =  number  of  measurements  in  error 

q  =  counter  from  1  to  d 
m  =  maximum  modeled  error 


Fixed  geometry  method 


if 


then  e 


25  equally  spaced  modeled  error 
values  between  zero  and  m 


Random  geometry  method 


5  equally  spaced  modeled  error 
values  between  zero  and  m 


Equations: 

if  q  <  d;  then  pjg  =  e 

Remarks :  Angular  measurement  error  are  iteratively  entered  into  the 

parameter  matrix. 

2  (Non-Common  Landmark  Misplacement) 

Definitions  of  Variables:  N/A 
Equations: 


P18 


e  10800 


/  2  2v 

n  IP12  +  Pi 3 ) 


1 

7 


Remarks:  See  appendix  D. 

3  (Common  Landmark  Misplacement) 
Definitions  of  Variables: 


0  is  number  of  landmarks  used  for  more  than 
one  measurement  (see  appendix  B) 


Equations: 


then 

Pi  8 


-e  10800 

2  2  ^ 

n  (Pi 4  +  Pi5)^ 

e  10800 

n  (Pi 2  +  Pi 3)^ 


Remarks:  See  appendix  D. 

4  (Range  Error) 

F  (Bearing  Error) 

F  (LORAN  Error) 

Definitions  of  Variables:  d  -  number  of  measurements  in  error 

q  =  counter  1  to  d 

Equations: 

if  q  <  d;  then  p^g  =  e 

Remarks:  Errors  are  entered  directly  into  the  parameter  matrix. 

T_  (Non-Common  Inclined  Angle  Error) 

Definitions  of  Variables:  N/A 


Equations: 


cos"ljc 


P18  =  60  cos”^\Cos (Pi6 )cos [  z  P16 


Remarks:  The  left  landmark  of  the  first  sextant  measurement  is 

elevated  (e)  meters  and  the  resulting  error  in  the  angular 
measurement  is  (pjg)  in  minutes. 

8  (Common  Inclined  Angle  Error) 

Definitions  of  Variables:  N/A 

Equations:  Same  as  7. 

Remarks :  The  error  is  the  same  as  for  Non-Common  Inclined  Angle  Error 

except  that  it  affects  two  angles  instead  of  one. 

9  (Observer  Coincidence) 

Definitions  of  Variables:  pjg  a  gradient  of  first  line  of  position 
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Equations: 

P18  =  e/pig 

Remarks :  This  error  simulates  the  displacement  of  one  observer  in  the 

direction  of  maximum  change  in  angular  measurement  (the  posi¬ 
tive  gradient  direction).  Studying  error  effect  by  using 
maximums  is  discussed  in  appendix  0. 

10  (S/0  Standard  Deviation) 

Definitions  of  Variables:  d  =  number  of  measurements  in  error 

q  =  counter  1  to  d 

Equations: 

(<d  (e 

if  q  <  then  p17  =  / 

(>d  (default  value  of  S/0  Standard  Deviation 

Remarks:  Sextant  measurement  s.d.'s  are  transferred  to  the  parameter 

matrix. 

11  (/  S/0  Standard  Deviation) 

Definitions  of  Variables:  i  *  1  to  n 

n  =  number  of  measurements 


Equations: 


if  i 


even  number  (modeled  error  value,  e 

then  p^7  =  < 

odd  number  (default  S/0  Standard  Deviation  value 


Remarks:  Standard  deviation  of  alternate  angular  measurements  are 

assigned  a  value  of  e  and  the  remaining  measurements  the 
default  s.d.  value.  This  logic  can  be  used  to  study  the 
effect  of  assuming  equal  s.d.'s  for  all  measurements  when  in 
fact  they  may  not  be  equal . 


12  (Non-Common  Landmark  Definition) 

Definitions  of  Variables:  dj  a  default  s.d.  of  first  measurement 

Equations: 


P17 


e  10800 


2  2l 
"■(P^  +  P13)^ 


1 


s.d.  of  the  perturbed 
measurement  in  minutes 


Remarks :  An  uncertainty  in  a  landmark's  position  adds  uncertainty  to 

the  measurements  made  using  the  landmark.  The  s.d.  of  the 
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position  of  the  landmark  along  a  line  perpendicular  to  the 
line  of  sight  from  the  observer  (which  represents  the  uncer¬ 
tainty)  is  weighted  by  the  distance  from  the  observer  to  the 
landmark.  The  resulting  s.d.  is  provided  by  the  equation 
above.  (See  appendix  D.) 

13  (Common  Landmark  Definition) 

Definitions  of  Variables:  N/A 


Equations:  Same  as  12. 

Remarks :  This  random  error  adds  to  measurement  s.d.  in  the  same  way  as 

Non-Common  Landmark  uncertainty  except  that  more  than  one 
angle  is  effected.  If  a  common  landmark  has  an  uncertainty 
in  its  position  perpendicular  to  the  observer's  line  of 
sight,  then  all  measurements  using  that  landmark  are 
effected.  If  both  landmarks  used  in  a  horizontal  angle 
measurement  are  poorly  defined,  then  both  contribute  to 
uncertainty  in  the  measurement. 

14  (Bearing  Standard  Deviation) 

15  (Range  Standard  Deviation) 

IF  (LORAN  Standard  Deviation) 

Definitions  of  Variables:  q  *  counter  from  1  to  d 

d  =  number  of  lines  affected 


Equations: 

if  q  <  d;  then  p^  *  e 

Remarks:  The  measurement  s.d.'s  are  transferred  into  the  parameter 

matrix,  replacing  the  default  s.d.  values  iteratively  through 
the  domain  of  the  modeled  error. 

17  (Vacant) 

18  (Vacant) 

GOOD 

Definitions  of  Variables:  N/A 

Equations:  N/A 

Remarks:  The  subroutine  OUTLIERS  calls  GOOD  if  an  error  effect 

parameter  calculated  for  one  of  the  random  fix  geometries  is 
two  s.d.'s  less  than  the  mean  of  all  previously  calculated 
error  effect  parameters  of  that  type.  GOOD  prints  parameter 
matrix  and  error  effect  parameter  values  for  further  study. 


BAD 


Definitions  of  Variables:  N/A 


Remarks : 


BAD  performs  the  same  function  as  GOOD  except  that  the  error 
effect  parameter  must  be  more  than  two  s.d.'s  greater  than 
the  mean  of  all  previously  calculated  error  effect  parameters 
of  that  type. 


FIXED 

Fa 

Fff 

FR 

Fl 

Definitions  of  Variables:  N/A 

Equations:  N/A 

Remarks :  FIXED  is  called  by  POSER  to  accept  user  definition  of  a  fix 

geometry.  It  accepts  ranges  and  true  bearings  to  all 
landmarks  which  are  part  of  the  unperturbed  system.  It  also 
accepts  LORAN  measurement  information.  FIXED  calls  FA,  FB, 
FR,  and  FL  as  necessary  to  define  the  fix. 


RANDOM 

RA 

m 

Rl 


Definitions  of  Variables:  G  *  number  of  random  geometries  to  be 

generated 

Equations:  N/A 

Remarks :  RANDOM  is  called  by  POSER  to  accept  user  input  of  the 

statistics  for  generation  of  random  geometries.  The 
statistics  used  in  the  ESM  are  derived  from  a  sample  set  of 
fix  geometries  from  the  USCGC  REDWOOD  (WLM  685).  Any  other 
sample  statistics  require  changes  to  RANDOM.  (See  appendix 
E.)  RA,  RB,  RR,  and  RL  are  called  as  necessary  to  establish 
statistics  for  generation  of  angular,  bearing,  range,  and 
LORAN  measurements,  respectively. 


PLOTR 

PLOTF 


Definitions  of  Variables:  Z$ 


ES 


*  a  string  array  of  output  parameter 
names 

■  a  string  array  of  error  names 

*  maximum  scale  value  of  the  pth  error 
effect  parameter 


Equations:  N/A 

Remarks :  PLOTR  and  PLOTF  create  the  axis  on  which  the  error  effect 

parameters  are  plotted  as  a  function  of  the  modeled  error. 
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APPENDIX  B 

ERROR  SENSITIVITY  MODEL  OPERATING  INSTRUCTIONS 


Prospective  "operators"  of  the  ESM  should  first  become  familiar  with  the 
modeled  errors  and  the  error  effect  parameters  before  attempting  to  operate 
the  routine.  Because  no  data  entry  error  feedback  mechanism  was  built  into 
the  routine,  the  instructions  should  be  strictly  followed. 

Listed  below  is  the  equipment  needed  to  operate  the  routine.  Familiarity  with 
the  operation  of  the  HP-9825A  is  assuned  throughout  this  appendix. 

a.  Hewlett-Packard  desk-top  calculator  HP-9825A  (Option  002,  23K  memory) 

b.  kRead  Only  Memory  (ROM)  String  Advanced  Programming  HP-92210A 

c.  ROM  HP-9872A  Plotter  General  I/O-Extended  I/O  HP-98216A 

d.  ROM  Matrix  HP-98211A 

e.  PI otter/Pr inter  HP-7245A  or  HP-9872A 

f.  Printer  HP-9866B 

g.  Interface  HP-IB98034A 


INSTRUCTIONS 

DISPLAY 

KEYSTROKE 

1. 

Insert  tape  cartridge 

2. 

Turn  on  calculator 

RUN 

3. 

The  calculator  displays 

#A,#B,#R,#L 

a.  ENTER  #A  (The  number  of  sextant  mea¬ 
surements  in  the  fix  geometry) 

CONTINUE 

b.  The  calculator  displays 

B? 

c.  ENTER  #B  (The  number  of  gyrocompass 
measurements  in  the  fix  geometry) 

CONTINUE 

d.  The  calculator  displays 

R? 

e.  ENTER  #R  (The  number  of  radar  mea¬ 
surements  in  the  fix  geometry) 

CONTINUE 

f.  The  calculator  displays 

L? 

g.  ENTER  #L  (The  number  of  LORAN  mea¬ 
surements  in  the  fix  geometry) 

CONTINUE 

INSTRUCTIONS 


DISPLAY 


KEYSTROKE 


4.  If  the  number  of  sextant  measurements  is 
greater  than  one,  the  calculator  displays 

a.  ENTER  #  COMMON  LM's  (The  number  of 
landmarks  used  for  more  than  one 
sextant  measurement  -  the  model  is 
limited  in  this  simulation,  the  first 
landmark  defined  is  always  a  non-common 
landmark,  then  all  common  landmarks  are 
defined,  then  the  final  landmark  is 
always  a  non-common  landmark} 

5.  The  calculator  displays 


#  COMMON  LM's 


CONTINUE 


ANY  NEW  DEFAULT 
STANDARD  DEVIATIONS? 


a.  ENTER  y  or  n  (n  denotes  the  default 
measurement  s.d.'s  are  adequate,  y 
signifies  that  the  default  measurement 
s.d.'s  are  inadequate  and  new  ones  are 
desired.)  The  calculator  displays 

(1)  Enter  desired  sextant  default 
value  (minutes) 

(2)  The  calculator  displays 

(3)  Enter  desired  qyrocompass  default 
value  (degrees) 

(4)  The  calculator  displays 

(5)  Enter  desired  radar  default  value 
(meters) 

(6)  The  calculator  displays 

(7)  Enter  desired  LORAN  default  value 
(jisec) 

6.  The  calculator  displays 


DEF.  VALS.? 


CONTINUE 


CONTINUE 


CONTINUE 


CONTINUE 


OUTPUT  PARAMETER 
NUMBER? 


ENTER  one  of  the  following  numbers 
for  error  effect  output  parameter: 

1  -  AP-to-MPP 

2  -  2-drms 

3  -  Maj*Min  (90*) 

4  -  Maj  (90*) 

5  -  P-in-R 

6  -  Sum  Sqd  Res 

7  -  Min/Maj 


CONTINUE 


1 


INSTRUCTIONS 


DISPLAY 


KEYSTROKE 


7.  If  output  parameter  number  5  (P-in-R)  was 
selected,  the  calculator  will  display 


a.  ENTER  the  radius  of  the  target  circle 
(meters ) 

8.  The  calculator  will  display 


a.  ENTER  y  or  n  (n  means  that  the  dimen¬ 
sions  of  plotting  grid  are  sufficient 
to  display  the  range  of  the  output 
parameter  selected,  y  means  that  the 
dimensions  are  not  sufficient.)  The 
calculator  will  display 

b.  ENTER  the  desired  range  of  the  output 
parameter.  (Trial  and  error  may  be 
needed  on  this  step  if  the  user  finds 
the  graphical  output  inadequate.)  The 
default  ranges  for  the  error  effect 
parameters  are: 

1  -  AP-to-MPP  =  100  meters 

2  -  2-drms  *  100  meters 

3  -  Maj*Min  (90%)  a  300  meters  squared 

4  -  Maj  (90%)  =  100  meters 

5  -  P-in-R  =  1.0 

6  -  Sum  Sqd  Res  =  25 

7  -  Min/Maj  *  1.0 

9.  The  calculator  will  display  ERROR  STUDIED? 

a.  ENTER  the  number  of  the  modeled  error  to 
be  studied: 

1  -  Index  Error 

2  -  NC  LM  Misplacement 

3  -  C  LM  Misplacement 

4  -  Range  Error 

5  -  Bearing  Error 

6  -  LORAN  Error 

7  -  NC  Inclined  Angle 

8  -  C  Inclined  Angle 

9  -  Observer  Coincidence 

10  -  ■  S/0  Standard  Deviation 

11  -  #  S/0  Standard  Deviation 

12  -  NC  LM  Definition 

13  -  C  LM  Definition 

14  -  Bearing  s.d. 

15  -  Range  s.d. 

16  -  LORAN  s.d. 

17  -  Not  used 

18  -  Not  used 


TARGET  CIRCLE 
RADIUS? 


NEW  OUTPUT  SCALE 
NEEDED? 


K(P)  ? 
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CONTINUE 


CONTINUE 


CONTINUE 


10.  The  calculator  will  display 


#  TIMES  IT 
OCCURRED? 


a.  ENTER  the  number  of  measurements  to 
which  this  error  is  being  subjected  to. 

(Consecutive,  starting  with  first  measure¬ 
ment  to  which  the  error  can  be  subjected)  CONTINUE 

11.  The  calculator  will  display  MAXIMUM  OF  ERROR 

STUDIED? 

a.  ENTER  the  maximum  of  the  error  studied 


(respective  units) 

CONTINUE 

The  calculator  will  display 

FIXED  GEOMETRY 
STUDIED? 

a.  ENTER  y  or  n  (y  means  that  the  fixed 
geometry  method  of  error  analysis  is 
desired,  n  means  that  the  random 
geometry  method  of  error  analysis  is 
desired.)  If  n  is  pressed,  the  cal¬ 
culator  will  display 

#  AND  SEED? 

b.  ENTER  #  and  SEED  (#  represents  the  num¬ 
ber  of  sample  geometries  desired  for  study 
of  the  error  effect  and  SEED  is  a  nine¬ 
digit  fraction  between  zero  and  one 
to  start  the  random  number  generator) 

CONTINUE 

c.  If  y  is  pressed,  the  calculator  will 
display 

GEOMETRY  NAME? 

CONTINUE 

d.  Enter  the  desired  geometry  name 
(This  will  label  output  graphs) 

CONTINUE 

e.  The  calculator  will  display 

R&B  TO  LEFT  OBJECT 

CONTINUE 

f.  ENTER  the  range  then  the  true  bearing  to 
the  left  landmark  of  the  first  sextant 
measurement  simulation 

r3? 

CONTINUE 

g.  The  calculator  will  display 

R&B  TO  RIGHT  OBJECT? 

CONTINUE 

h.  ENTER  the  range  then  the  true  bearing  to 
the  right  landmark  of  the  first  sextant 
measurement  simulation 

r3? 

CONTINUE 

i.  Repeat  f,  g,  and  h  for  all  sextant  measure 
ments 


CONTINUE 


INSTRUCTIONS 


DISPLAY 


KEYSTROKE 


j.  The  calculator  will  display  R&B  TO  OBJECT  (B) 

k.  ENTER  the  range  then  the  true  bearing 

to  landmark  used  in  gyrocompass 
measurement  s i mu 1  at i on  r 3 ? 


l.  Repeat  k  for  all  gyrocompass  measure¬ 
ment  simulations 

m.  The  calculator  will  display  R&B  TO  OBJECT  (R) 

n.  ENTER  range  then  the  true  and  bearing 

to  landmark  used  in  radar  measurement 
simulation  r3? 


o.  Repeat  n  for  all  radar  measurement  simu¬ 
lations 


p.  The  calculator  will  display 

q.  ENTER  the  LORAN  gradient  then  the 
magnitude 

r.  Repeat  q  for  all  LORAN  measurement 
simulations 

13.  The  calculator  will  display 


GRAD  DIR  AND 
MAGNITUDE?  (L) 

P  (1,3)? 


PAPER  IN  PLOTTER? 


a.  Press  CONTINUE  when  the  plotter  is 
set  to  plot 

14.  The  calculator  will  plot  the  desired 
grid  and  error  effect  parameter  in  the 
fixed  geometry  case 

15.  The  calculator  will  begin  the  time- 
consuming  study  of  a  random  sample  of 
geometries  in  the  random  geometry  method. 

Following  completion,  the  calculator  will 

display  CUMULATIVE? 

a.  ENTER  y  if  a  cumulative  frequency  plot 
are  required 

b.  ENTER  n  if  frequency  distribution  is 
required 


CONTINUE 

CONTINUE 

CONTINUE 

CONTINUE 

CONTINUE 

CONTINUE 

CONTINUE 

CONTINUE 

CONTINUE 

CONTINUE 


CONTINUE 


INSTRUCTIONS 


DISPLAY 


KEYSTROKE 


16.  The  calculator  will  display  PAPER  IN  PLOTTER? 

a.  Press  CONTINUE  when  the  plotter  is 
set  to  plot 

17.  The  calculator  will  print  out  all  plotted 
values  and  label  the  rows  and  columns  with 
appropriate  headings  for  future  reference 

18.  The  calculator  prints  END  OF  RUN 


CONTINUE 
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APPENDIX  C 


NUMERICAL  INTEGRATION  BY  GAUSSIAN  QUADRATURE 
(Reference  14) 


Frequent  use  of  the  P-in-R  parameter  in  the  ESM  dictates  the  necessity  for  a 
mathematical  description  of  the  method.  A  brief  discussion  of  some  mathemati¬ 
cal  aspects  of  the  numerical  method  is  made  in  this  appendix  leaving  the  more 
complicated  explanations  to  reference  14. 

Many  numerical  methods  for  integration  of  known  functions  call  for  equally 
spaced  points  within  the  region  of  integration  at  which  the  integrand  is 
evaluated.  Gaussian  Quadrature  employs  points  of  unequal  spacing  within  the 
region  and  is  the  preferred  scheme  for  numerical  integration  for  this  applica¬ 
tion.  The  selected  points  are  zeros  of  orthogonal  polynomials.  The  degree  of 
the  polynomials  is  determined  by  the  desired  accuracy  of  the  integration. 

The  concept  is  to  evaluate  an  integral  by  selection  of  the  formula: 


n 

5Z  Aiy(xi ) 

i  =1 


(C-l) 


Where:  x^  =  unequally  spaced  zeros  of  orthogonal  (Legendre  Polynomials) 
polynomi als. 

A,  =  weighting  values  determined  by  orthogonal  polynomials. 

The  details  of  calculating  the  x-j  and  Ai  values  are  found  in  reference  14. 

For  ESM  application,  the  sixteen-point  Gaussian  Quadrature  is  adequate.  The 
sixteen-point  method  requires  evaluation  of  the  integrand  at  256  points  within 
a  two-dimensional  region  of  integration.  When  multiplied  by  the  model  error 
value  and  geometry  iterations,  a  huge  number  of  calculations  are  required 
consuming  much  computer  time.  However,  by  comparison,  the  Gaussian  Quadrature 
scheme,  for  equivalent  accuracy,  is  at  least  twice  as  fast  as  any  other  scheme. 


Errors  occur  when  the  integrand  changes  rapidly  over  the  region  of  integra¬ 
tion.  In  the  ESM  this  error  appeared  when  the  s.d.'s,  which  define  the 
bivariate  probability  density  function,  were  less  than  20%  of  the  radius  of 
the  circular  region  of  integration.  If  the  P-in-R  parameter  proves  to  be  a 
candidate  as  a  measure  of  success  in  aid  positioning,  more  study  will  be 
needed  to  define  the  tradeoff  between  speed  and  accuracy. 


The  calculation  of  P-in-R  within  the  ESM  is  performed  by  the  subroutine  PROBA¬ 
BILITY.  PROBABILITY  evaluates  the  integral, 


DIS(x,y)  dx  dy 


16 


£ 


A-jAj  DIS(xityj)  (C-2) 
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Where: 


The  weighting  values  are  the  same  as  described  above  and  the 
(x^y,-)  points  of  integration  are  located  within  the  region  of 
interest  (reference  14).  R  is  the  radius  of  the  circular  region  of 
integration. 

The  calculations  were  verified  by  comparison  with  table  Q-6-c  in  appendix  Q  of 
Bowditch  (see  reference  2). 

The  numerically  evaluated  integrals  were  within  0.1%  of  being  equal  to  the 
tabulated  values. 
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APPENDIX  D 


CONSEQUENCES  OF  STUDYING  ERRORS  ONLY  IN  THE  DIRECTION  OF  MAXIMUM  EFFECT 


Within  the  ESM,  the  following  systematic  modeled  errors  could  have  been 
studied  as  displacement  vectors: 

a.  Non-common  landmark  misplacement 

b.  Common  landmark  misplacement 

c.  Observer  coincidence 

Furthermore,  landmark  definition  could  have  been  studied  using  bivariate 
probability  distributions. 

However,  studying  each  of  the  above  modeled  errors  as  vectors  or  bivariate 
probability  distributions  would  add  another  dimension  as  well  as  mathematical 
complications  to  the  error  modeling  task.  The  discussion  in  this  appendix 
verifies  the  legitimacy  of  studying  the  component  of  the  displacement  vector 
in  the  direction  of  maximum  affect  and  verifies  the  use  of  a  univariate 
probability  distribution  in  studying  Landmark  Definition. 

The  position  P  is  determined  by  resection  with  L  as  one  of  the  reference  land¬ 
marks.  The  effect  landmark  displacement  has  on  the  position  P  is  determined 
by  the  effect  it  has  on  the  lines  of  position  which  were  determined  using  L  as 
a  landmark.  The  effect  displacement  has  on  the  lines  of  position  is  deter¬ 
mined  through  use  of  the  gradient  equation  which  is  (reference  3,  appendix  B): 

D  *  G  dO  (D-l) 

Where:  D  3  Distance  LOP  is  displaced 

G  ■  Gradient  of  the  line  of  position 
dO  3  Small  angular  change  in  measurement 

For  all  practical  situations  the  gradient  remains  constant  in  the  region  of 
interest.  That  is,  when  the  displacement  magnitude  Is  small  compared  to 
distances  between  landmarks  and  P.  The  angular  measurement  change  due  to 
landmark  displacement  is  therefore  the  only  important  quantity. 

Figure  D-l  depicts  a  representative  landmark  displacement  error  vector  show¬ 
ing  its  effect  on  an  angular  measurement.  The  symbology  in  the  figure  is  as 
follows: 

E  3  Magnitude  of  displacement  vector 
P  3  Position  of  observer 
R  3  Range  from  observer  to  landmark 
L  3  Landmarks  position  without  error 
Lj  3  Landmark  position  with  projected  displacement 
L2  *  Landmark  position  with  vector  displacement 
Ep  3  Magnitude  of  projected  displacement  vector 
a  3  Angle  of  displacement  vector  with  line  perpendicular  to  line 
of  sight  CPC) 

dOj  3  Angular  error  due  to  projected  landmark  displacement 
dQ?  3  Angular  error  due  to  displacement  vector 

D-l 
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It  can  be  seen  from  figure  0-1,  that  studying  displacement  perpendicular  to 
the  line  of  sight  (PL)  Is  a  close  approximation  to  studying  the  displacement 
vector  when  considering  the  angular  change  d6.  A  condition  for  this  approxi¬ 
mation  Is  that  E  be  much  smaller  than  R,  which  Is  the  case  for  most  displace¬ 
ment  of  landmarks.  The  need  for  this  condition  Is  shown  as  follows: 

when  E  <  <  R 


sin  dQj 
By  the  law  of  sines 


Ecos  a 

(E2cos2  a  +  r2)*4 


Ecos  a 

— r~ 


(D-2) 


„  _  E  $1n(180-(90+  a  *dQ2))  _  u„a 

sin  002  * - b  ~  — — b — 


(D-3 ) 


Therefore,  sin  dO^  «  sin  d  02  ,  and  dO^  »  d02  and  It  Is  legitimate  to 
study  the  projection  of  the  displacement  vector  upon  the  line  perpendicular  to 
the  line  of  sight.  It  Is  now  possible  to  extend  the  results  of  studying  the 
projected  displacement  error  to  results  concerning  the  displacement  error 
vectors . 


The  direction  of  the  displacement  vector  is  uniformly  distributed  in  all 
directions  from  the  landnarks  true  location.  The  distribution  which  repre¬ 
sents  the  magnitude,  E,  Is  unknown  and  unimportant  for  only  the  effect  of  E  on 
P  is  modeled  in  the  ESM.  For  all  values  of  E,  a  is  uniformly  distributed. 

For  the  first  quadrant  the  probability  density  function  for  a  Is: 


f(a)  U(0,  y-)  ( Simi liar ly  for  all  other  quadrants)  (D-4) 


The  projected  value  of  E  Is  Ep  «  E  cos  a 


Averaging  both  sides  of  the  equation  over  the  respective  intervals  will  yield 
the  expected  value  of  Ep  when  the  error  E  and  directions  are  known. 


2  E 

<Ep>*  E  <cos«>  *  n 


da 
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This  result  Indicates  that  studies  within  the  ESM  which  involve  any  of  the 
three  systematic  modeled  errors  listed  at  the  start  of  this  appendix  represent 
conservative  estimates  using  maximum  error  effect.  A  better  measure  of  the 
average  effect  due  to  a  displacement  vector  of  magnitude  E  can  be  calculated 

2  F 

by  using  a  modeled  error  of  magnitude  ■=»■—  .  This  gives  results  which  differ 
by  a  factor  of  £  for  those  errors  which  propagate  linearly.  A  similar 

argument  can  be  made  for  landmark  definition  as  It  can  be  considered  a 
displacement  of  the  observed  landmark  coordinates  from  the  true  (horizontal 
control)  coordinates. 
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APPENDIX  E 

DISTRIBUTIONS  FOR  RANDOM  GEOMETRY  METHOD 


The  statistics  used  in  generating  random  geometries  were  calculated  from 
historical  data  on  fixes  taken  by  the  crew  of  USCGC  REDWOOD  (WLM  685).  The 
following  procedures  were  followed: 

a.  SANDS  forms  were  researched. 

b.  A  sample  (n  «  100)  of  each  of  the  following  variates  was  taken  from 
the  data: 

(1)  Range  to  non-common  landmarks 

(2)  Range  to  common  landnarks 

(3)  Angles  between  landmarks 

(4)  Gradients 

(5)  Line  of  position  crossing  angles 

c.  The  mean  and  s.d.  of  the  first  three  variates  were  calculated  and 
the  distributions  were  tested  for  normality. 

The  Information  compiled  was  used  to  derived  the  following  results: 

a.  The  distribution  of  ranges  to  common  and  non-common  landmarks  were 
not  significantly  different. 

b.  The  ranges  had  a  mean  of  4000  meters  and  a  s.d.  of  2000  meters,  and 
were  distributed  as  a  truncated  normal  distribution  (truncated  at 
zero  meters).  The  effect  of  this  truncation  is  insignificant  in  the 
generation  of  fix  geometries. 

c.  The  angles  had  a  mean  of  60°  and  a  s.d.  of  35°  and  were  distri¬ 
buted  as  a  truncated  normal  distribution  (truncated  at  0°).  The 
effect  of  this  truncation  is  Insignificant  in  the  generation  of  fix 
geometries. 

d.  The  gradients  generated  by  the  Monte  Carlo  Routine  were  distributed 
Identically  with  those  of  the  sample  set. 

e.  For  a  fix  with  n  lines  of  position  the  n-1  smallest  crossing  angles 
are  uniformly  distributed  between  zero  and  180°/n. 

The  statistics  derived  from  the  sample  allow  random  generation  of  fix  geome¬ 
tries  for  the  sextant,  gyrocompass,  and  radar  In  the  following  way: 

a.  Sextant  -  The  routine  generates  a  simulated  landnark  with  a  range 
distributed  as  per  sample  statistics  and  with  a  bearing  uniformly 
distributed  around  the  horizon.  An  angle  Is  generated  using  sample 
statistics  and  Is  followed  by  another  range.  The  three  variates 
together  simulate  the  geometry  of  one  sextant  angle.  Other  simu¬ 
lated  measurements  of  the  same  fix  are  similarly  generated.  If  two 
angles  use  a  common  landmark,  the  right  landnark  of  the  first  mea¬ 
surement  Is  used  as  the  left  landmark  of  the  second. 
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Gyrocompass  -  The  routine  generates  a  simulated  landmark  with  a 
range  distributed  as  per  sample  statistics  and  with  a  bearing  uni¬ 
formly  distributed  around  the  horizon.  Other  simulated  gyrocompass 
measurements  of  the  same  fix  are  generated  with  a  crossing  angle 
uniformly  distributed  between  30°  and  90°  of  the  previous  simu¬ 
lated  measurement  of  the  geometry. 

Radar  -  Generates  a  simulated  landmark  In  same  manner  as  with 
gyrocompass  simulations. 

LORAN  -  The  routine  generates  crossing  angle  in  the  same  manner  as 
for  both  the  gyrocompass  and  radar  routines.  As  a  first  approxima¬ 
tion,  LORAN  gradients  are  generated  with  a  truncated  normal  distri¬ 
bution  with  a  mean  of  300  meters//isec  and  a  standard  deviation  of 
150  meters/psec  (truncated  at  150  meters/jjsec).  The  effect  of  this 
truncation  will  slightly  raise  the  mean  gradient  by  discarding  all 
values  less  than  150  meters/jjsec  and  regenerating  the  gradient 
needed.  This  distribution  was  created  by  studying  a  random  sample 
of  LORAN  grids  on  Mercator  projection  navigation  charts  (sample  size 
n  -  100). 


APPENDIX  F 

DISCUSSION  OF  OUTLIER  DETECTION 


A  method  to  determine  whether  or  not  a  given  set  of  measurements  conform  to 
normal  expectations  Is  discussed  In  this  appendix.  Before  any  method  can  be 
successfully  employed  as  a  part  of  a  calculator-based  positioning  system,  the 
following  Items  will  require  further  consideration. 

a.  Assumed  measurement  variances 

b.  Conditions  unique  to  specific  aid  locations 

c.  Combinatorial  analysis  of  fix  situations 

d.  Independence  of  measurements 

e.  Post  blunder  detection  procedures 

A  discussion  In  reference  11  provided  the  basis  for  the  outlier  detection 
method  presented  here. 

The  computed  values  of  measurements  for  the  desired  location  are  assumed  to  be 
the  parent  population  means  of  each  measurement.  Ideally,  assuming  no  sys¬ 
tematic  error,  the  measurements  are  normally  distributed  about  the  parent 
population  means.  The  differences  (1*)  In  the  measurements  from  the  parent 
population  means  are  normally  distributed  variates  with  a  mean  of  zero  and  a 
variance  of  <rf,  where  <r\  Is  the  measurement  s.d.  The  error  detection 
method  employs  the  xz  probability  distribution  which  Is  Introduced  as 
follows.  The  sum  of  the  squares  of  n  Independent  random  variables  having 
standard  normal  distributions  has  the  xjj  distribution  where  n  denotes  the 
degrees  of  freedom.  The  variates  If/**  form  a  standard  normal  distribu¬ 
tion  (reference  11). 

Thus: 


2 

The  sun  of  n  normalized  squared  errors  are  xn  distributed.  This  Is  equivalent 
to  the  x 2  "goodness  of  fit"  test.wlth  no  parameters  being  estimated  by  the 
sample  measurements.  With  the  xz  method,  any  set  of  the  measurements  can  be 
checked  for  agreement  with  the  computed  values  at  some  desired  confidence 
level, a . 

For  any  subset  of  m  measurements ,  the  test  Is, 


(  «  ) 


(F-2) 


The  confidence  level  chosen  Is  dependent  upon  the  geometry  of  the  fix  and 
criticality  of  the  aid  (reference  1)  being  positioned. 


F-l 


I 


2 

If  the  sun  exceeds  the  x  m  ,  inconsistency  of  the  measurements  relative  to 
the  assumed  measurement  variance  Is  Indicated  and  one  or  more  of  the  following 
possibilities  exist  concerning  the  measurement  subset, 

a.  The  position  determined  by  the  measurement  subset  Is  not  the  desired 
position. 

b.  Blunders  and/or  uncompensated  for  systematic  errors  exist  In  one  or 
more  of  the  measurements. 

c.  A  chance  (1-a)  outlier  situation  occurred. 

In  case  a,  subsequent  maneuvering  should  allow  Improvement.  In  the  event  of 
case  b,  the  measurements  in  question  should  be  Investigated.  Possibilities 
include  individual  checks  of  each  measurement,  measuring  Instruments,  and  of 
signal  sources  for  accuracy.  An  example  of  the  xz  method  Is  as  follows:  a 
vessel  Is  maneuvered  by  "marking  two  measurements"  and  the  error  In  a  third 
measurement  Is  checked  against  some  prespecified  xz  value.  If  the  observa¬ 
tion  error  exceeds  x\  at  some  prespecified  confidence  level,  a, an  Inves¬ 
tigation  of  the  all  three  measurements  Is  in  order. 

A  mathematically  more  complicated  test  of  the  residuals  of  a  set  of  n  measure¬ 
ments  can  be  performed  using  a  similar  procedures.  In  this  procedure,  the 
measurements  are  made,  a  most  probable  position  Is  determined  from  the 
measurements,  and  the  xz  test  Is  performed  on  the  weighted  residuals.  The 
formula  Is, 


t  (£-) 2  »  **  <*> 

1«1  v  1  ’  n-2 

Where  the  n-2  is  the  number  of  degrees  of  freedom  resulting  from  the  loss  of 
two  degrees  in  the  estimation  of  the  MPP  from  the  measurement  set. 

If  the  test  Indicates  Inconsistency,  the  following  possibilities  exist: 

a.  The  measurement  geometry  is  functionally  inconsistent. 

b.  One  or  more  measurements  are  in  error  and  exceed  the  confidence 
limits  Imposed  by  the  assuned  <r. 

Case  a  requires  redundant  precomputation  or.  If  that  fails,  a  new  geometry. 
Case  b  requires  Investigation  of  the  measurement  set  (all  measurements). 
Possibilities  here  also  Include  study  of  measuring  Instrunents  and  signal 
sources  for  accuracy.  Further  statistical  tests  can  be  performed  on  subsets 
of  the  measurement  set.  If  four  or  more  lines  are  used,  the  sum  of  the  norma¬ 
lized  squared  residuals  of  each  three-measurement  subset  with  n-3  degrees  of 
freedom  can  be  compared  to  the  normalized  squared  residual  of  the  nth  measure¬ 
ment  with  reference  to  the  MPP  determined  by  the  n-1  measurement  subset.  The 
normalized  squared  residual  of  the  nth  measurement  has  one  degree  of  freedom. 
The  test  Is  for  n(>3)  measurements: 


F-2 


Where:  F(l,  n-3, a)  is  the  F  distribution  with  1  degree  of  freedom  In  the 

numerator  and  n-3  degrees  of  freedom  In  the  denominator* 

This  calculation  would  be  repeated  for  each  subset  of  (n-1)  measurements  with 
the  largest  of  the  ratios  tested  for  significance  against  the  desired 
F-statlstlc.  A  significant  result  would  indicate  that  the  corresponding  nth 
measurement  is  likely  In  error. 


APPENDIX  G 

ERROR  SENSITIVITY  MODEL  LISTING 


0:  no  “POSER" 

1:  “SN":  rad5r<-21n<rnd<l>>>cos<2rfrndCl>>*Pl5de-35  ret  p1 

2s  ,'DIS*sexp<-.5<plPl/p3p3+p2p2xp4p4>>/'2ifp3p4*p55  ret  p5 

3:  "DEFAULT": for  1  =  1  to  N5for  J=1  to  4?  if  PC  I » 1  3=J5  DC  J 3+PC  I » ?  3 

4:  next  J 5  next  1 5  ret 

5:  “GENERATE" :0*r4+r5*rS 

6s  for  1=1  to  N 

7s  *sb  "Mfl“ I  if  PC  It  1  3>1 Jesb  “NB,,5if  PCI,i3>25?sb  "HR" 5  if  PCI,13>35*sb  “NL 
•8  s  next  1 5  ret 
9s  "NR" s 

10s  if  Q#0  and  I>1  and  I <=0+1 j Jmp  3 

lls  MC I, 1,  1 D+’SN’MC I, l*2]+rl5 if  rl<0  or  rl >160005 no  +0 
12s  rnd< 1 >*368+r25  rlsin<  r2)+PC 1 , 2  35 r lcos<  r2)+PC I >  3  35  jwp  2 
13s  PC  I  —  1 » 4  3+PC  I,  2  35 PC  1-1,5  3+PC  1,3  3 

14s  MC 1,2, 1 3+*SN»MC I,2,23+rl5  if  rl<0  or  . rl >16000! no  +8 

15$  MC  1 , 3, 1 3+-' SN* MC  1,3,2 3*r3+PC  1,635  if  r3<0  or  r3>1885no  +0 

16s  r2+r3+r25  rlsin<r2>+PC  1,4  35  rtcos<r2>+PC 1,53 

17s  if  r2>3605 r2-360+r2 

18s  ret 

19s  "MB“s  l  +  r4+r*4 

20s  MC  1, 1, 1 3+’SN’MC  1, 1,2  3+rlJ  if  rl<0  or  rl>lS0005  no  +8 

21  s  if  r4<25  rnd< 1 )*368+r2+PC  1,6  35  j«p  2 

22$  r2+rnd<l)*60+30+r2+PC 1,63 

23s  if  r2>3685 r2-360+r2 

24s  rlsin<r2)+PC  1,2  35  rlcos<r2>+PC  1,3  35  ret 

25:  "HP": l+r5+r5 

26s  MC  1 , 1 ,  1  3+’  SN’  MC  1 , 1 , 2  3+rl+PC  1,6  35  if  rl<0  or  rl  >168805  no  +8 

27s  if  r5<25 rnd<l>*360+r2! j«P  2 

28:  r2+rnd< 1 > *60+30+ r2 

29:  if  r2>3605r2-360+r2 

30:  rlsin<r2>+PC  1,235  rlcosC r2>+PC  1 , 3 35  ret 

31$  “HL"si+r6+r6 

32$  it  r6<25 rnd<l)#360+r25 jwp  2 

33s  r2+rnd< 1 >*60+30+r2 

34:  if  r2>3685r2-360+r2 

35$  r2+PC  1,23 

36:  MC  1, 1,  1  3+’SN’MC  1, 1,23+rlJ  if  rl<l50  or  rl>15005no  +8 
37s  rl+PC  I,3  35  0+PC  1,6  35  ret 
38$  "GRADIENT" s 
39$  for  1*1  to  H 

40:  PC  1 , 1 3+rl  5PC 1 , 2  3+r25PC  1 , 3  3+r3?  PC  1 , 4  3+r45  PC  1 , 5  3+r5 

41 s  esb  "GR" 5  if  rl>l?«b  "GBM5if  rl>2!  ?sb  "GR"5if  rl>35  ?sb  "GL" 

42$  if  rl*l  and  PC  1 , 9 3>1005 if  *  35*sb  “GENERATE" 

43$  if  fl435cfe  35*to  -4 
44 s  next  IJret 
45*  “Gfl"S 

46:  r < <  r2r2+r3r3><  r4r4+r5r5> > .  00029xr <  < r2-r4> t2+< r3-r5>T2>+PC 1,93 

47:  <  r 3< r4r4+r5r5)-r5<  r2r2+r3r3> )/2< r4r3-r2r5>+r6 

48$  <  r2<r4r4+r5r5)-r4<  r2r2+r3r3> )/2< r2r5-r4r3>+r7 

49$  if  r6=8  and  r7>05 8+PC  1,  10 35 jwp  4 

*6627 
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5 U s  it  r6=0  and  r7<05 180*PC I » 10 ]! jnP  3 
5J  :  :  t  r6 >0 5  90-at n < r?/ r6 > *K  I>  1 0  3 5  •  d mp  2 
52:  it  rS<  05 270-atn< r7/r6>*PC  I » 10  ] 

53:  ret 

•  H  «  • 

55:  r<r2r2+r3r3>.017453*PC  1*93 

56:  PC  I»63-90*PC  I»  1035  if  PCM0K85PC  I »  10  3+360+PC  1*10  3 
57:  ret 
53:  "GR": 

59:  1 ->PC I « 9  3 

60:  if  r2=0  and  r3>05 180*PC  I >  10 3!  jmp  4 
61:  if  r2=0  and  r3<05  0*PC  I  >  10  35  JNP  3 
62:  if  r2<05  90-atn<r3/r2XPCI» 1035 jmp  2 
63:  if  r2>05 270-atn< r3/r2)+PC I » 10 3 
64:  ret 

65:  “ GL “ :  PC  I » 3  3*PC  I » 9  3  5  PC  I » 2 3*PC  I » 1 0  3  5  ret 
66:  "PARTIALS”: 

67:  for  1=1  to  N 

68:  -PC  I >  3  3+BC  21-1 35 -PC  I > 5  3*BC  21 3 
69:  -PC  I  *  2  3*CC  21-1  35  -PC  I  >  4  3+CC  21  3 

70:  r  <  BC  2 1  - 1  3t2+CC2I-l  3t2)*SC2I-l  35  r  <BC  21  3T2+CC2I  3t2>*SC2I 3 

71:  next  I 

72:  for  L=1  to  N 

73:  9sb  "API" 5  if  PC  L»  1  3>1 5  <ssb  "RB"5if  PC  L,  1 3 >25  esb  " fiR M I  if  PC  L»  1 3>35  *sb  MAL' 
74:  next  L?  ret 
75:  “Ah": 

76:  2L-1+I52L+J 

77 :  CC  I  3/SC  13X1  /SC  J  3-cos  < PC  L »  6  3  >  /SC  I  3  >  *AC  L  X  3 

73 :  AC  LX  3+  < CC  J  3/SC  J  3 X 1  /SC  1 3-cos < PC  L » 6  3  > /SC  J  3 > *RC  L »  1  3 

79:  -AC  L<  1  3/s in<PC  L*  6  3.XRC  LX  3 

80 :  BC  I  3/SC  13X1  /SC  J  3-cos  <  PC  L »  6  3  >  /SC  I  3  >  *AC  L  >  2  3 

8 1 :  HC  L  f  2  3+ 1  BC  J  3/SC  J  3  >  <  1  /SC  I  3-cos  <  PC  L  j  6  3  > /SC  J3>*ACL»23 

82:  -AC  L*  2  3/s inC PC  L»  6  3 )  *RC  L  j  2  3 

S3:  ret 

84:  "hB": 

35:  if  PC  L  j  3  3=0  5  0*  AC  L  *  1  3  5  j  ftp  2 
86:  -co*<PCL»6  3>t2/PCL»3  3*ACL»  1 3 
87:  if  PC  L  j  2  3=05  0"fAC  L  j  2  35  Jf'iP  2 
88 :  si n < PC  L j  6  3 ) t2/PC  L j  2  3+AC  L » 2  3 
89:  ret 

C|£^  •  "  j^p  "  ■ 

91:  if  PCLj  63=050*ACLX  35-jnp  2 
92:  -PC  Ln  2  3/PC  L>  6  3*AC  LX  3 
93:  if  PC  L»  6  3=05  0* AC  L j  2  35  jmp  2 
94:  -  FT  L  3  3/PC  L  >  6  3+ AC  L  *  2  3 
95:  t  et 
“ ftL  '  » 

97 :  si n < PC  L j  2  3 > /PC  L >  3 3*RC  L  X  3 
98 :  cos  <  PC  L »  2  3 >  /PC  L  >  3  3+ AC  L  j  2  3 

99:  ret 
*7663 


•  0-2 


r 


■SI 


1  tjy 

101 

102 

10? 

104 

105 

106 
107 

.108 

109 

110 
111 
'112 

113 

114 

115 

116 
117 


OUTLIERS" s 
for  J=1  to  7 

p  J+OC  p8>  J*  1  3*0C  p3i  Jj  1  35  PJT2+QC  p8»  J»  2  3*^0C  p8»  Jj  2  3 

next.  j:  if  p9<305ret 

for  J=1  to  7 

DC  p8»  Jj  1  3/p9+0Cp8>  Jj3  3 

r ( < 0C  pS > J > 2 3/P9-OC p8»  J » 3 3t2 > p9x < p 9- 1 )  >*GC  p3>  J>  4  3 
if  J=5  or  J=75 Oi-iP  3 

if  pj>0t p3> J> 3 3+20C p8» J? 4 3  and  fl?l=@5esb  "BAB" 

if  pJ<0C p8» J» 3 3-20C p8j Jj 4 3  and  flel=8  and  p8#85-3sb  "GOOD" 

next  JJret 

” PROBABILITY" s  sf  e  145 -p4cos(p1 >-p5sin<Pl )*p155  p4sin<Pl )-p5cos(p1 >*p5 
if  2r<p2p2+p3p3>+r(p5p5+pl5t2><p65 ret  1 
if  r<p5p5+pl5t2>-2r<p2p2+p3p3) >p65 ret  0 

for  1  =  1  to  165 0*p95 P6IC I j 1 3+p15*p105 r<p6p6-<Pl0-Pl5> t2)*Pl 1 
for  J=1  to  165ICJ>23,DIS,<p10>p11ICJ>1  3+p5» p2> p3>+p9*p9 
next  J5 p9IC I > 2 3p1 1+p14*p145 next  1 5  p  14p6*p145  cf  •?  145  ret  p14 
“  1" : 


118 

119 

120 
121 
122 


0*Q 

for  1=1  to  N 

if  PC  I  j  1  3=1 5  <3+1*05  if  Q<=B5  E*PC  I  j  3  3 
next  1 5  ret 

»!  It  a 

Ex-  :PC  1  *  2  3T2+PC 1  ■  3  3t2>*10808/if*PC  1 » 


3 


124 

125 

126 
127 
123 
1 29 
138 

131 

132 

1 33 

134 

1 35 

1 36 

137 

138 

139 

148 

141 

142 
1  4? 

144 

145 

146 

147 

143 

149 
*23 


ret 

"3": 

for  1=1  to  N 

if  PCI? 13=1  and  1=1 5 -E/r<PC I > 4 3t2+PC I j 5 3t2>*< 10880/u >*PC I j 8 3 
if  PC  I  >13=1  and  I  =0+1 5  E--T  <  PC  I »  2  3t2+PC  I  j  3  3T2  >  *  <  1 0808-'ff  >  *  PC  I  j  8  3 
next  1 5  ret 
"4“  ! 

0*Q 

for  1  =  1  t  o  N 

if  PC  I j 1 3=35  Q+1*Q5 if  GK=D5E*PC I j  8  3 
next  1 5  ret 

cr  ••  ■ 

.J  s 
0*0 

f o r  1=1  to  N 

if  PC  I .« 1 3=25 Q+1*Q5 if  Q<=D5 E*PC I > 3 3 
next  1 5  ret 
"  c  "  • 

0*Q 

for  1=1  to  N 

f  PC  I  f  1  3=4  5  Q+ 1  *0  5  i  f  GK  =B  5E*PC  I?  83 
next  1 5  ret 


acs  \  cos  f  PC  1 1 6  3 ) cos 


E 1 80-T  <  PC  1 »  2  3T2+PC  1  j  3  3T2  >  n  > 


■PC  1 j  6  3 > 6 8 * P C 1 


ret 

•*  rt  »•  • 

O  • 

for  1=1  to  N 

21 


0-3 


it  PC  If  1 3#i; j«P  5 

if  I #  1 SJ rip  2 

.  v.  i~.  ■  ;  '  .  r.  M  _  f  P  C  I  ?  5  3  C  O  S  f*. 

if  I#0+15 j«p  2 
i  acs  ( co  5-  t  PC  1*6]  ^  cos  c 
ne  xt  1 5  ret 


:  E 1 80.  T  PC  1.4  3t2+PC  1 , 5  3t2  >  if  >  > -PC  1.6]  >  6©*PC  1.83 
:  E 1 80  T<  PC  1 . 2  3 1 2+PC  1 . 3  3t2 )  if  >  >  -PC  1.63)  60*PC  1.33 


E-  'F’C  1 . 9  3*PC  1.83 
ret 
"  10": 

Q*Q 

for  1=1  to  N 

if  PC  I.  1  3=1  5Q+1+Q5  if  Q<=D> E-fPC  1.73 
next  I .ret 
"11" : 

f o r  1  =  1  to  N 

if  PC  1. 13=1  and  int < I/2)=0. E*PC I > 7  3 
next  I j  ret 
••12": 

r  < PC  1 . 7  3f  2+  < E 1 080©/r  <  PC  1 . 2  312+PC  1 . 3  3t2 >  if  >  t2 > *PC  1.73 
ret 
"  13": 

for  1  =  1  to  N 

if  PC  I  ?  1  3#  1 5  J ivip  6 

if  1  =  1 5  r<PC  1 . 7  3t  2+  <  E 1 0800/ r  ( PC  I.43T2+PC  1 .5  3t2>ir>t2>+PC  1.73! 
i  f  I  =0+ 1 5  r  <PC  1 . 7  3T2+  <  E 1 0800/r  <  PC  1 . 2  3t2+PC  1 . 3  3t2  >  if  >  1 2 )  *PC  1 . 7 
if  I  >0+1 ;  jfiP  3 

r  ( PC  1 . 7  3 1 2+  < E 1 0800/r  <  PC  1.2  3+2+PC  1 . 3  3t 2  )  if)t2>*PC  1.73 
r  PC  1 . 7  3t2+ < E 1 0800/r < PC  1 . 4  3t2+PC  1 , 5  3t2 >  if  >  t£  >*PC  1.73 

next  I? ret 
"14”: 

0+Q 

tor  1  =  1  t  o  N 

if  PC  I »  1 3=2 5  Q+l-tQ.  if  Q<*DSE+PC  1.7  3 
next  I J  ret 

**  l  C  "  • 

1  J  ■ 

0*Q 

for  1=1  to  N 

if  PC  1 . 1  3=3 5  Q+ 1  *Q  j  i f  Q<  =Ii  j  E*PC  1.73 
next  I .ret 


if  PC  1 .  1  3=4  5  Q+1*Q5  if  0<  =  D 5  E-tPC  1.73 
next  I 
r  e  t 

"GOOD"  :  wrt  65  sf  ?  15  if  f  l  =i8#l  5  wrt  6.2 
wrt  6. 3*  wrt  6.4 
for  1=1  to  N 

t  o r  J= 1  to  1 0 5  PC  I . J  3* r J 5  next  J 


OnP  5 
3  5  i  ivi  p  4 


*23540 


247: 

248: 

24'?: 

*305 


w rt  6 . 5, r  1 ,  r2 ,  r3 ,  r 4  ,  r 5 »  r6 ,  r 7 ,  r8 ,  r9 »  r  1 0 

,  T 

sift.  65  wr t-  6. 65  wrt  6.7 

wrt  6.  5, E-C,  2C  1 3,  ZC  2  3,  ZC  3  3, 2C  4  3,  ZC  5  3,  ZC  6  3,  2C  7  3 

ret 

"BAD": wrt  65  sf*  15  if  fl*0#15wrt  6.1 
urt  6. 3?  wrt  6.4 
for  I = 1  to  N 

for  L=1  to  105  PC  I ,  L  3* rl_ 5  next  L 
w  rt  6 . 5 ,  r  1 ,  r2 ,  r3 ,  r 4 ,  r5 ,  r 6 ,  r7 ,  r8  ,  r9  »  r  1 0 
next  I 

wrt  65 wrt  6. 65 wrt  6.7 

wrt  6. 5, E-C,  ZC  1  3 ,  ZC  2  3,  ZC  3  3,  ZC  4  3 ,  ZC  5  3.  ZC  6  3,  ZC  7  35  ret 
“FIXED" : sf 9  0 
“FR“ : 

for  1=1  to  R 
1*PC I ,  1 3 

enp  "R&B  to  LO" ,  r2» r35 r3*r4 

r2s  i  n  <  r3  >  *PC  1,235  r  2  c  o  s  ( r 3 )  *PC  I  ,  3  3 

enp  “R&B  to  RO" , r2 ,  r35  r3-r4+360*PC I , 6  3 

if  PC  I »  6  3 >360 5  PC  1 , 6 3-360+PC I , 6 3 

r2i-  i  n  <  r3  >  *PC  I  ,  4  3  5  r  2cos  <  r  3  >  *PC  1,53 

next  I 

"FB": 

for  I=A+1  to  R+B 
2*PC  1,13 

enp  "RSsB  to  Object  ( B )  "  ,  r2,  r35  r 3 * P C  1,63 
r2i-  i  n  <  r3 )  *PC  1,23!  r 2 c o s  <  r-3  >  *PC  1,33 
next  I 
"FR": 

for  I=R+B+1  to  R+B+R 
3*  PC  1,13 

e np  " R&6  t o  Object  < R ) " ,  r 2 ,  r*3 5  r2*PC  1,63 
r2s  i  n  <  r3  >  *PC  1,2  35  r 2 c o s  (  r3 )  *PC  1,33 
next  I 

"FL": 

for  I=fl+B+R+1  to  N 
4*PC 1,13 

enp  "Grad  dir.  and  ivio.-e.  <L> " ,  PC  1 , 2  3,  PC  1 , 3  35  0*PC  1 , 6  3 

next  I 

ret 

"RANDOM”: 

enp  “ #  and  Seed? " ,  G »  r3 !  wo. i t  rnd < -r3 > 

"Rfl" : 

for  1  =  1  t  o  R 
1*PC I,  1  3 

i  f  0 #0  a n d  I  >  1  a. n d  I  <  =0+1 5  J  m p  2 
4000*MC  1,1,13!  2000*MC  1 ,  1,23 
4000*MC  1,2,  1  35  2000*MC 1,2,23 
60*MC  1,3,  1  35  354MC  1,3,2  3 
26 


T 


299: 

*2646 


"RB  ' : 

for  I=A+1  to  A+B 
2+ PC  III] 

4000*HC  I  ,  1  ,  135  2000+MC  1,1,23 

next  I 
"RR“: 

for  I=R+B+1  to  A+B+R 
3+PC 1,13 

4000+MC  1 1  1 »  1  35  2000-tMC  1,1,23 

next  I 

"RL“: 

for  I=A+B+R+1  to  N 
4+PC  I  j  1  3 

300+MC  1 1  1*  1  35  150*MC  1 1 1 » 2  3 

next  I 

ret 

"PLOTR": 

wrt  705 » " IP000» 000>  6000 , 3000“ 

sc  1  0,9,0,65 xax  0»0,0»95yax  0,0,0»65xax  6,0»0»95yax  9»@»0,6 
csiz  2, 1.7,  2/35  pit  4,. 2? lbl  Z*CP3 

fxd  05 pit  7. 75i 3. 55 lbl  "A=  ”51bl  A5Plt  7. 75, 3. 255 lbl  "B=  “51bl  B 
Pit  7.75,35  lbl  "R*  "5  lbl  RJplt  7. 75, 2. 755  lbl  "L=  ”5  lbl  N-<A+B+R> 
pit  2 »  5. 755  lbl  "Random  Geometry" 5  Pit  2> 5. 55 lbl  "Number®" 5 lbl  GJfxd  1 
Pit  4. 75 j  5. 755  if  caP<A$>="Y"5 lbl  "Cunul  .ms.  "!J«p  2 
lbl  "Fract  .ms.  " 
lbl  Z*CP3 

Pit  4.75,5.55  lbl  E«  r&  3 

Pit  4.75,5.255 lbl  "MAXIMUM®  "5 lbl  M 

if  f  l-s25  Pit  4.75,55  lbl  “P=  "5  lbl  F5  lbl  "  meters" 

csiz  2,  1 , 2/3,905  pit  .3,25  if  cap<A$>="Y" 5 lbl  "Cimulat iMe" 5 j«p  2 

lbl  "Fraction" 

csiz  2,1, 2/3 

wrt  705, " IP1000, 500  ,5000,2300" 
sc  1  0,  KC  P  3, 0, 1 

xax  0, KC P 3/10, 0, KC P 3, 25 yax  0,. 1,0, 1,2 
csiz  2, 1 . 7, 2/35  ret 
"PLOTF" : 

wrt  70S, " IP000, 000, 6000, 3000" 
sc  1  0,9, 0,6 

sc  1  0,9,0,65 xax  0,0,0,95 yax  0,0,0,65 xax  6,0,0,95 v ax  9 , 0 , 0 , 6 

csiz  2,  1.7, 2/35  Pit  4,. 25  lbl  E*Cr03 

Pit  2, 5. 755  lbl  "Fixed  Geowet rv" 5 pit  2, 5. 55 lbl  N* 

Pit  4.75,5.755  lbl  Z*CP35  1bl  "  .ms.  "5  lbl  E$C r0 3 

Pit  4.75,5.55 lbl  "MAXIMUM®  "5 lbl  M 

if  f 1325 Pit  4.75,5.255  lbl  "R=  "5 lbl  FJlbl  "  weters" 

csiz  2,  1 , 2/3, 905 pit  .4,251bl  2*1  P 3 

wrt  705, " IP1000,  700,5000,2500" 

sc  1  0,  M, 0, KC  P  3 

fxd  25 xax  0, M/10, 0, M, 2? yax  0,KC P 3/10, 0, KC P 3, 2 


>e "  5  J  I'l p  2 


6,0,0,95  v  a  x  9 , 0 , 0 , 6 


T 


I 


r 


i-j  y 

:  .•  e  t 

301 

:  "CHANGE": enp  "Def 

Va 1 s?A *  B 

>  R  *  L  " 

302 

"PO 

SER" : t  rk  8 

303 

frit 

1* "Bad  Outlier"*/ 

384 

frit 

2 *" Good  Outlier"*.' 

385 

f  fit 

3*  4x * "TYPE 

XL 

YL 

386 

t  mt 

4*"  BIAS 

GRAD 

PGD" 

387 

fmt 

5* 10f 3. 2 

383 

fmt 

6* "  E 

2 

2RMS 

309 

f  mt 

7*"  RES 

RATIO" 

310 

fmt 

8 *  4x * "BIN 

3 

.  2M 

311 

fmt 

9>"  MEANS 

0 

.  2M 

312 

eriP 

"#A>  #B*  #R>  #L" 

*  A*  B*  R*  LJ 

if  A> 

313 

:  A+B+R+L*N 

DC  1  ]>  DC  2 : 

XR 

]*  DC  3  ]* 

YR 

DC  4  1!  ret 

MEAS 

SIGMA 

AREA 

MAJ 

P  IN  R  " 

*  2 

.  4M 

.  6M 

.  8M 

M"*/ 

.  4M 

.  6M 

.  8M 

M", 

2! enp  “# 

Common 

LM’s?">0 

314 

315 

316 

317 
313 


dim  AC  N*  2  3*  BC  2N  3  *  CC  2N3*  DC  4  3*  EC  2*  N  3*  FC  0:  5  *  26  3  *  HC  2*  £  3*  It  16*  2  3*  KC  7  3*  LC  N  3 
dim  MCN*3*23*N[  2*23*  OC  0:5*7*43*  PC  N*  10  3*  QC  2*  2  3*  RC  N  3*  SC  2N  3*  TC  2*  N  3 
dim  VC  2  *  2  3 >  WC  N >  N  3  *  XC  2  3 *  YC  2  3 *  2C  7  3 *  A$C  1  3*  E$C  20*  20  3*  N$C  15  3*2*C7>20  3 
ldf  l,E$!ldf  2*2$51df  3*IC*3 

1004KC  1  35  100*KC  2  35  300*KC  3  3*  100+KC4  3! 1->KC  5  3J  50+KC  6  3!  1+KC7  3 


319 

320 

321 


5*DC  1  3?  .5*DC23530*DC335  .  1*DC43 

enp  “Any  New  Default  Std.  Deu.?y  or  n"*A$5if  caP<A$)*"Y" ! esb  "CHANGE 
enp  "Output  Parameter  Number?" * P5 prt  2$CP3 


322 


if  P=5*enp  “Target  Circle  Radius?<meters> ” * F5 sf e  2 


•_i  £.  -1' 


324 


0*3 


327 


enp  "New  Output  Scale  Needed?/  or  n"*A$ 

if  cap<fi$)a"Y“ 5 enp  "New  Output  Scale  Value0" *  KC  P  3 

enp  "Error  studied?" > r0> enp  "#  Times  it  Occurred?"  *  D*  prt  E$Cr8  3 

enp  "Maximum  of  Errors  studied?" >M 

enp  "Fixed  Geometry  Method?/  or  n“*A$ 

esb  "RANDOM  "5  if  ca»<A*)»"Y"  5  er.p  "Fixed  Geometry  Name?" * N$! esb  "FIXED 
if  f 1 305 f xd  15dsp  "Paper  in  Plotter?" 5 stp  >9sb  "PLOTF" 
if  f 1905 M/25*C> wrt  6.65wrt  6.7»  imP  4 
M/3+C 

for  K=1  to  G 
?sb  "GENERATE" 
ssb  "GRADIENT" 


3  6‘  5 

336 

337 
333 
33? 

340 

341 

342 

343 
3^4 

345 

346 

347 
343 
349 


Cf'9  1 

for  E=0  to  M  by  C 
9sb  "DEFAULT" 
"ERROR": 


no  +1?  if  r0>55  n o  +3?  if  r0>18!-«o  +5?  if  r0>145eto  +7 

ssb  '  1  ' 5  i f  rO>l  j  9sb  ”2” 5  if  r0>2>esb  "3“ 5  if  r8>35esb  "4"  5  if  r9>4l«b  " 

jmp  6 

9sb  "6";  if  r0>6?  9sb  "7"?  if  r8>7!  ssb  "3"?  if  r0>85ssb  "9"!  if  r0>9i«b  " 
jmp  4 

ssb  "11"! if  r0 >115 ssb  "12"! if  r0>12!ssb  "13"!  if  r0>13!ssb  "14" 
jmp  2 

ssb  "15  !  if  r0>i5!ssb  "16"!  if  r8>l6!ssb  "17"!  if  r0>175ssb  "13" 
ssb  "PARTI ALS" 

"WEIGHTS": 
for  1*1  to  N 


-*18633 


•  0-7 


250 :  it  Ft  I  f  7  3=05  1  0t6 *WC  I  >  I  3  5  .i nP  5 
351:  if  PC  l,  i  3=1  S  C108Q0/PC  I»7H>t2+UC  I*  I] 

352:  if  PC  I » 1  3=25  ( 180-  PC  I » 7 3ii  >  t2+MC  I  >  1 3 
353:  if  PC  I  j  1  3=3?  1/PC  I  »  7  3t2+MC  lull 
354:  if  PC  I>  1  3=45 1/PC  I»?3t2+WC  I»  I] 

355:  next  I 
356:  BIAS'  : 

357:  for  1=1  to  N 

353:  if  PC  I  >  1 3=  1 5  -PC  I  *  3  3if/ i 0800+LC  I  3 
359:  if  PCI>13=25-PCI»8  3ir/180+LCI3 
360:  if  PC  I > 1 3>=35 -PC  I » 8 3+LC I  3 
361:  next  I 
362:  "REDUCE": 

363:  trn  A+T  5  nat  TW+E5wat  EA+V5  nat  EL+X5  ino  V+NSnat  NX+YJsnpv  -1Y+X 
364:  nat  AX+R 
365:  ara  R+L+R 

366:  if  HC  1  >  1 3-NC 2> 2 3=0  and  NC  1>23<05  135+T5 Jm»  6 
367:  if  NC 1. 1 3-NC 2> 2 3=0  and  NC 1 » 2 3=05 0+T5 dnp  5 
363:  if  NC  1  ?  1 3-NC  2j  2  3=0  and  NC  1 , 2  3>05  45+T5  J.v,p  4 
369:  .  Sat n < 2NC 1  >  2  3/ <NC  1  >  1 3-NC2»2  3>>+T 
370:  if  NC  1  >  1 3-NC  2»  2  3<05  T +90+T 
371:  if  TC05T+130+T 

372:  cos<T>*VC 1» 1  3-^VC  2»  2 35 sir.<T>+VC 2»  1 35 -VC  2*  1  3+VC  1,2  3 
373:  tr n  V + Q * m a t  Q N + H  5 m a  t  H  V *  0 
374:  “ZOUTPUT" : 

375 :  r < XC 1 3 1 2+XC 2  3t2 > +ZC  1  3 

376:  2r<QC  1,  1  3+QC2,2  3>+ZC2  3 

377 :  2 .  1 5t2r <  QC  1 »  1 3QC  2 »  2  3  > *ZC  3  3 

373 :  2.1  Srnax <  QC  1 , 1  3,  QC  2 ,  2  3)  +ZC  4  3 

379:  rpiaxCQC  1,  1 3,  QC  2, 2  3)  +  r  1 1 5  JViinCQC  1»  1  3,  QC  2,  2  3>+rl2 

330 :  if  f  l *2 5  *  PRGBfiB I L I  TV »  <  T ,  r  1 1 ,  r  1 2 » XC  1  3,  XC  2  3 ,  F  >  +ZC  5  3 

33 1 :  0+ZC  6  3 5  f  o r  1  =  1  to  N 5  RC  I  3t2HC  I  j  1 3+ZC  6  3+ZC  6  35  next  I 

332 :  im  i n <  r  1 1  j  r  1 2  >  /n ax <  r  1 1 »  r  1 2  >  +ZC  7  3 

383:  if  f I  30 5 i mp  5 


334 

385 

336 

387 

339 

390 

391 

392 

393 

394 

395 

396 

397 

398 

399 
*219 


oil  *  OUTLIERS’ <ZC 1 3  >  ZC  2  3 ,  ZC  3  3  >  ZC  4  3 » ZC  5  3 ,  ZC  6  3 ,  ZC  7  3 »  E/C ,  K  > 
"TABULATE": 

int  <ZC  P  3*25-'KC  P  3+1 i  +  r65  if  r6>25?  26*r6 
FCE/Cj r6 3+1 +FC E/C » r6 3 

if  i  1  '?0 'pit  E,  ZC  P  35  wrt  6.5,E»2Cl3,ZC23»2C33,Zt43»Zt53»2C63»ZC?3 
next  E 

if  f  1-905  Prt  "End  of  Run"  5  9Sb  “GOOD" 
if  f 1*05  end 
next  K 
"PLOTOUT": 

enp  "Cunulat ive?" , A$ 

txd  1 5 dsp  "Paper  in  Plotter?" 5 stP  5 esb  "PLOTR" 
fxd  0 

for  1  =  1  to  265  for  J=0  to  55  FC  J,  I  3/G+FC  J,  I  35  next  J 5  next  I 
if  cap  (A#  )*"Y"  5  jr-iP  3 
for  J=1  to  5 
:'6 


0-8 


400 

401 

402 

403 
4G4 

405 

406 

407 

408 

409 

410 

411 

412 

413 

414 

415 


for  1  =  1  to  25*  Pit  ( I  - .  5  >  *Kt  P  3-  25  >  Ft  J  >  I  3*35  1  b  1  JJr.ext  1 5  next  JiJnP  3 

for  J=  1  t  o  5*  for  1  =  1  to  24$0*rlifor  H=  1  to  I  >  Ft  J*  H  3  +  r  1  -> r*  1 5  next  H 

Pit.  (I-.5)*Kt P ]/25*  rl *  3* lbl  JJnext  I*Plt  .93>l*35  1bl  JJnext  J 

"PRINTOUT'S 

wrt  65wrt-  6.8Jwrt  6 

for  1*1  to  26 

wrt  6.5*  <I-.5)KtP]/25>FC0*  I  3, Ft  1*  I  3* Ft  2*  I  3*  Ft  3*  1 3*  Ft  4, 1  3*  FC  5*  I  3 

next  I 

wrt  6* wrt  6 

for  J=3  to  4 

if  J=35wrt  6.9ij«p  2 

wrt  6.5* "STD  DEV  0  .  2M  . 4M  .6M  .  8M  M" 

wrt  6 

for  1  =  1  to  7 

wrt  6.5*  I»OC0>  I*  J3*0t  1*  I*  J3*OC2*  I*  J3*OC3*  I»  J3*OC4*I*  J3*OC5*  I*  J3 
next  IJwrt  6* next  JJend 


*14322 


E$ 


Z  $ 


Index  Error(nin) 
NC  LM  MisP(net ) 
C  LM  Nisp(net) 
F: o. nee  E  r  r o  r  <  ne  t  > 
Bearine  Error(dee) 
LORflN  Error (usee) 
NC  Incl  fine (met) 
C  Incl  fine (net) 
Observer  CoinCnet ) 
=S  '0  Std  Dev (n in) 
#S/0  Std  Deo(nin) 
NC  LM  Def (net ) 
C  LM  Def (net ) 
Bearine  Std  Deo (dee) 
Ranee  Std  Deo (net ) 
LORfiN  Std  Dew (usee) 


fiP  to  MPP(net) 
2DRMS(net ) 
M  j  *Mn  ( 98% )  ( nt  t2 ) 
Mad  (90*;>  (net ) 
Prob  in  R 
Sun  Sad  Res 
M  i  rv  Ma  J 


It  *  3 


-0.9894Q0930 
-0, 9*4575020 
-0. 865631200 
-0.755404410 
-0.617876240 
-0.458016780 
-0.281603550 
-0.095012510 
0.095012510 
0.281603550 
0.458016780 
0.617876240 
0.755404410 
0.865631200 
0.944575020 
0.989400930 


0.027152460 
0.062253520 
0.095158510 
0. 124628970 
0. 149595990 
0. 169156520 
0. 182603420 
0. 189450610 
0. 189450610 
0. 182603420 
0. 1691 56520 
0. 149595990 
0. 124628970 
0.095158510 
0.062253520 
0.027152460 


